4

I would like to merge two non-git-based text files using semantics that are similar to to how git depicts "merge conflicts".

For example, suppose I have two text files with similar but not identical content, called file.1 and file.2. I would like to perform a merge of these two files into a third file, as follows:

hypothetical-merge-utility file.1 file.2 file.merged

I would like it to produce file.merged, which would list the file contents and each and every diff in a manner similar to this:

common line 1 ...
common line 2 ...
common line 3 ...
<<<<<<< file.1
something unique from file.1
a second line of something unique from file.1
======= file.2
something unique from file.2
>>>>>>> end of diff
common line 4 ...
common line 5 ...
<<<<<<< file.1
something unique from file.1
======= file.2
something unique from file.2
a second line of something unique from file.2
>>>>>>> end of diff
common line 6 ...
common line 7 ...
... etc. ...

In other words, I'd like each diff between file.1 and file.2 to look similar to the representation of a git "merge conflict".

I don't care if delimiters other than <<<<<<<<, ========, and >>>>>>>> are used.

I know that there are a number of utilties available for merging text files under linux. However, I am only looking for something which specifically presents the merged data in a manner similar to the the way that git depicts "merge conflicts".

Does anyone know of such a utility?

Thank you in advance.

UPDATE: Per the question below from Ed Morton, here are the contents of the two test files ...

==== file.1 ====

common line 1 ...
common line 2 ...
common line 3 ...
something unique from file.1
a second line of something unique from file.1
common line 4 ...
common line 5 ...
something unique from file.1
common line 6 ...
common line 7 ...

==== file.2 ====

common line 1 ...
common line 2 ...
common line 3 ...
something unique from file.2
common line 4 ...
common line 5 ...
something unique from file.2
a second line of something unique from file.2
common line 6 ...
common line 7 ...
1
  • Thank you. I edited the question as requested. Now the contents of the two test files, file.1 and file.2, are shown.
    – HippoMan
    Sep 18, 2022 at 16:55

3 Answers 3

2

NOTE: Although I consider this to be a somewhat reasonable "Answer", I've now come up with another "Answer" which I think is better. So please see my other "Answer", below.

Original version of this "Answer" ...

Oh! I posted too soon here. I wasn't aware of the -D command-line option to diff, and now I realize that I can do this ...

diff -D file.1 file.2 >file.merged

It will produce the following insde of file.merged ...

common line 1 ...
common line 2 ...
common line 3 ...
#ifdef file.1
something unique from file.1
a second line of something unique from file.1
#else /* file.1 */
something unique from file.2
#endif /* file.1 */
common line 4 ...
common line 5 ...
#ifdef file.1
something unique from file.1
#else /* file.1 */
something unique from file.2
a second line of something unique from file.2
#endif /* file.1 */
common line 6 ...
common line 7 ...
... etc. ...

I can deal with the #ifdef, #else, and #endif lines in the same way as I can deal with git's <<<<<<<<, ========, and >>>>>>>> lines.

UPDATE: ... and I just found this: https://stackoverflow.com/questions/16902001/manually-merge-two-files-using-diff

It shows how I can also do something similar with unified diff format. Give diff a -U option with a huge argument which is larger than the maximum number of lines in file.1 and file.2. For example ...

diff -U 99999999 file.1 file.2 | tail -n +4 >file.merged

It will then produce this:

 common line 1 ...
 common line 2 ...
 common line 3 ...
+something unique from file.2
-something unique from file.1
-a second line of something unique from file.1
 common line 4 ...
 common line 5 ...
+something unique from file.2
+a second line of something unique from file.2
-something unique from file.1
 common line 6 ...
 common line 7 ...
 ... etc. ...

The + lines represent the unique data in file.2, and the - lines represent the unique data in file.1.

And I can deal with those + and - lines.

3
  • Regarding I can deal with the #ifdef, #else, and #endif lines in the same way as I can deal with git's <<<<<<<<, ========, and >>>>>>>> lines. - none of that can be relied on for identifying the source file since any of those strings could be present in any input file. Regarding diff -U 99999999 - what you show isn't the only output that'll produce so you then have to parse that too.
    – Ed Morton
    Sep 18, 2022 at 17:03
  • The values <<<<<<<<, ========, and >>>>>>>> can also be strings in the file. However, when git shows merge conflicts, it also uses those strings in exactly the same way. I totally accept the use of <<<<<<<<, ========, and >>>>>>>> to depict git merge conflicts and can deal with them, and therefore, I also totally accept the use of those #ifdef, #else, and #endif in the same way.
    – HippoMan
    Sep 18, 2022 at 17:12
  • As for the diff -U 99999999 output, all the lines are shifted one position to the right with a blank in the leading position, except for the +, and - lines, which start in the leftmost position, so in that way, the diff -U output is less ambiguous and in that sense, more desirable. And yes, I left out the topmost 3 lines from diff -U. I'm adding a tail command to the example to take care of that case.
    – HippoMan
    Sep 18, 2022 at 17:17
0

It sounds like you don't REALLY care about the output format and instead just want to know how to identify which lines are from each file or are common. Give that, how about:

$ diff --old-line-format=$'-%l\n' --new-line-format=$'+%l\n' --unchanged-line-format=$'=%l\n' file.1 file.2
=common line 1 ...
=common line 2 ...
=common line 3 ...
-something unique from file.1
-a second line of something unique from file.1
+something unique from file.2
=common line 4 ...
=common line 5 ...
-something unique from file.1
+something unique from file.2
+a second line of something unique from file.2
=common line 6 ...
=common line 7 ...

Be wary of any solution where you have to test the content of the lines for an indicator of the source for that line (e.g. if you're looking for <<<<<<< file.1 to tell you what's unique to file1 - what if file contains a line that is exactly that string?) rather than an indicator that is always and only present in a unique position in each line since testing for any string will fail if that string could be in your input. With the above the first character is always an indicator of where the line came from so it cannot clash with possible file contents. If you REALLY wanted to get exactly the git merge conflicts format of output (which I don't recommend) you could always pipe the above to a simple awk script to print <<< file or whatever you like when the first character of the line changes and then remove that character.

1
  • Yes, this format does indeed resolve ambiguities. But one thing I do not want to do is deal with all the =common lines which remain in the merged output. Of course, once I manually resolve the diffs in the merged file I can do a follow-up edit of that file with sed or my own text editor to get rid of ^common (regex format). But I'd prefer not to do that. I'm well accustomed to dealing with git "merge conflct" output which doesn't do anything to the common lines, and I'm happy to use something that gives the same or similar output.
    – HippoMan
    Sep 18, 2022 at 17:24
0

Due to the limitations of the solutions I originally posted in my first "Answer" here which involve diff -D ... and diff -U ..., I decided to write a solution in python, using python's difflib module.

I wrote it to produce output which looks relatively similar to the "merge conflict" output from git. It utilizes delimiters containing the strings <<<<<<<<, ========, and >>>>>>>>, and as we know, this might lead to ambiguities if the original text contains strings like this. However, this same problem with ambiguities could exist within "merge conflict" output from git, but since I'm comfortable with git and willing to accept it there, I'm also comfortable with these ambiguities in my own solution.

The output isn't precisely the same as that from git "merge conflict" output, but it's similar enough to satisfy my wishes.

First, here is the python program (I cleaned up the original python code that I posted here, and this is the cleaned-up version). I call this program filemerge ...

#!/usr/bin/python3

### Take the diff's between two files and output
### the common and different lines in a manner
### which is very similar to the way that `git`
### depicts merge conflicts.

import sys
sys.dont_write_bytecode = True

import os

from difflib import unified_diff

prog       = None
diff_start = '<<<<<<<<'
diff_sep   = '========'
diff_end   = '>>>>>>>>'

def main():
    if len(sys.argv) < 3:
        print(f'\nusage: {prog} file1 file2\n')
        return 1

    file1, file2 = sys.argv[1:3]
    data1        = None
    data2        = None
    missing      = []

    try:
        with open(file1, 'r') as f:
            data1 = f.readlines()
    except Exception:
        missing.append(file1)

    try:
        with open(file2, 'r') as f:
            data2 = f.readlines()
    except Exception:
        missing.append(file2)
        
    if missing:
        print(f'\nnot found: {", ".join(missing)}\n')
        return 1

    n1 = len(data1)
    n2 = len(data2)
    max_lines = (n1 + 1) if n1 > n2 else (n2 + 1)
    count = 0
    state = ''
    sep_printed = False
    next_file = ''

    for line in unified_diff(data1, data2, n=max_lines):
        count += 1
        if count < 4:
            continue

        # Every line which is returned by unified_diff()
        # is at least 2 characters long. Each of these
        # lines starts with either ' ', '+', or '-', and
        # each of these lines ends with a newline.
        line = line[:-1]
        ch0  = line[0]

        if ch0 == ' ':
            if state:
                state = ''
                if not sep_printed:
                    print(f'{diff_sep}{next_file}')
                print(diff_end)
            sep_printed = False
            next_file = ''
        elif ch0 == '-':
            if state == ch0:
                pass
            elif state == '+':
                print(f'{diff_sep} file={file1}')
                sep_printed = True
                next_file = ''
            else:
                print(f'{diff_start} file={file1}')
                sep_printed = False
                next_file = f' file={file2}'
            state = ch0
        elif ch0 == '+':
            if state == ch0:
                pass
            elif state == '-':
                print(f'{diff_sep} file={file2}')
                sep_printed = True
                next_file = ''
            else:
                print(f'{diff_start} file={file2}')
                sep_printed = False
                next_file = f' file={file1}'
            state = ch0
        print(line[1:])

    if state:
        if not sep_printed:
            print(f'{diff_sep}{next_file}')
            next_file = ''
        print(diff_end)

    return 0

if __name__ == '__main__':
    prog = os.path.basename(sys.argv[0])
    sys.exit(main())

Here are the input files that I tested it with. They're similar but not exactly the same as the input files I originally posted in my question here ...

==== file.1 ====

common line 1 ...
common line 2 ...
common line 3 ...
something unique from file.1
a second line of something unique from file.1
common line 4 ...
common line 5 ...
something unique from file.1
common line 6 ...
common line 7 ...
penultimate file.1 line
common line 8 ...

==== file.2 ====

common line 1 ...
second line from file.2
common line 2 ...
common line 3 ...
something unique from file.2
common line 4 ...
common line 5 ...
something unique from file.2
a second line of something unique from file.2
common line 6 ...
common line 7 ...
common line 8 ...

I run the command like this ...

filemerge file.1 file.2 >file.merged

And these are the resulting contents of file.merged ...

common line 1 ...
<<<<<<<< file=file.2
second line from file.2
======== file=file.1
>>>>>>>>
common line 2 ...
common line 3 ...
<<<<<<<< file=file.1
something unique from file.1
a second line of something unique from file.1
======== file=file.2
something unique from file.2
>>>>>>>>
common line 4 ...
common line 5 ...
<<<<<<<< file=file.1
something unique from file.1
======== file=file.2
something unique from file.2
a second line of something unique from file.2
>>>>>>>>
common line 6 ...
common line 7 ...
<<<<<<<< file=file.1
penultimate file.1 line
======== file=file.2
>>>>>>>>
common line 8 ...

As I mentioned, this isn't precisely the same format as that of the "merge conflict" output from git, but it's very similar, and that's close enough for me.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .