Using a regex search with sed to sort entries in a file

Question

I have a text file of the following form:

dir1/sub-dir1/.../filename1 author date
dir1/sub-dir1/.../filename2 author date
.
.
.
dir2/sub-di2/.../filename1 author date
dir2/sub-dir2/.../filename2 author date

Additional details:

• Every filename listed exists precisely 1 time or 2 times. That is, filename_n exists precisely 1 time or 2 times. If it exists 1 time, it is in only 1 version of our software, v1 or v2, depending on the path. If it exists 2 times, the file is in both v1 and v2.
• The path determines if the file is in v1 or if it is in v2.
• The text file has is tab delimitted.
• The number of subdirectories varies from file to file (hence the ... in the code block.)
• (The authors and dates were not taken from ls. They were generated separately, by interrogating the git commits that last recorded changes to these files.)

I am trying to use sed with a regex to transform the file so that it looks like this:

dir1/sub-dir1/.../filename1 author date dir2/sub-di2/.../filename1 author date
dir1/sub-dir1/.../filename2 author date dir2/sub-dir2/.../filename2 author date
.
.
.

If filename-n does not have a match, then it should appear only once, not twice.

I'm looking for a way to use sed and regex to perform the transformation. It can either be written to the same file (with -i), or it can be written to a separate file.

Answer 1

One could do this entirely with sed, but this would result in a complex script which is hard to comprehend, so I would suggest to sort the file by the filenames first and then use sed to join lines if a filename is found twice.

The filenames are in the third subdirectory, to we tell sort to use the fourth field (-k 4) with the \ (why on earth!!) as field separator:

sort -t'\' -k 4 /tmp/p|sed 'N;/\(\\[^\]*        \).*\1/s/\n/    /;P;D'

The sed command uses the typical N;P;D cycle to always process two lines at once and check whether the filename (between \ and TAB) gets repeated.

Please note that that there are two literal tabs in the sed scripts. With GNU sed, you could write \t instead:

sort -t'\' -k 4 /tmp/p|sed 'N;/\(\\[^\]*\t\).*\1/s/\n/\t/;P;D'

Also note that paths including TAB characters or backslashes can break the script.

Answer 2

Using any awk in any shell on every Unix box no matter which characters your paths contain (other than newlines):

$ cat tst.awk
BEGIN { FS=OFS="\t" }
{
    file = $0
    sub(".*/","",file)
    paths[file] = (file in paths ? paths[file] OFS : "") $0
}
END {
    for ( file in paths ) {
        print paths[file]
    }
}

$ awk -f tst.awk file
dir1/sub-dir1/.../filename2 author date dir2/sub-dir2/.../filename2 author date
.       .       .
dir1/sub-dir1/.../filename1 author date dir2/sub-di2/.../filename1 author date

If the output order actually matters it's a trivial tweak - just tell us what the output order should be (e.g. first read, alphabetical by file name, alphabetical by directory, something else).