0

I have a text file of the following form:

dir1/sub-dir1/.../filename1 author date
dir1/sub-dir1/.../filename2 author date
.
.
.
dir2/sub-di2/.../filename1 author date
dir2/sub-dir2/.../filename2 author date

Additional details:

  • Every filename listed exists precisely 1 time or 2 times. That is, filename_n exists precisely 1 time or 2 times. If it exists 1 time, it is in only 1 version of our software, v1 or v2, depending on the path. If it exists 2 times, the file is in both v1 and v2.
  • The path determines if the file is in v1 or if it is in v2.
  • The text file has is tab delimitted.
  • The number of subdirectories varies from file to file (hence the ... in the code block.)
  • (The authors and dates were not taken from ls. They were generated separately, by interrogating the git commits that last recorded changes to these files.)

I am trying to use sed with a regex to transform the file so that it looks like this:

dir1/sub-dir1/.../filename1 author date dir2/sub-di2/.../filename1 author date
dir1/sub-dir1/.../filename2 author date dir2/sub-dir2/.../filename2 author date
.
.
.

If filename-n does not have a match, then it should appear only once, not twice.

I'm looking for a way to use sed and regex to perform the transformation. It can either be written to the same file (with -i), or it can be written to a separate file.

7
  • Are filenames unique even without paths, i.e. is there no name that exists in different directories?
    – choroba
    Apr 21 at 22:18
  • "Every file listed exists either 1 time or 2 times." :-)
    – JoshG
    Apr 21 at 22:20
  • OK, so things like v1/config.yml and v1/subproject/dir/config.yml don't happen.
    – choroba
    Apr 21 at 22:27
  • Exactly. What will happen frequently, though, is v1/../config.yml and v2/../config.yml.
    – JoshG
    Apr 21 at 22:52
  • Is there is reason for backslashes instead of slashes to separate directories in paths?
    – Philippos
    Apr 22 at 6:35

2 Answers 2

0

One could do this entirely with sed, but this would result in a complex script which is hard to comprehend, so I would suggest to sort the file by the filenames first and then use sed to join lines if a filename is found twice.

The filenames are in the third subdirectory, to we tell sort to use the fourth field (-k 4) with the \ (why on earth!!) as field separator:

sort -t'\' -k 4 /tmp/p|sed 'N;/\(\\[^\]*        \).*\1/s/\n/    /;P;D'

The sed command uses the typical N;P;D cycle to always process two lines at once and check whether the filename (between \ and TAB) gets repeated.

Please note that that there are two literal tabs in the sed scripts. With GNU sed, you could write \t instead:

sort -t'\' -k 4 /tmp/p|sed 'N;/\(\\[^\]*\t\).*\1/s/\n/\t/;P;D'

Also note that paths including TAB characters or backslashes can break the script.

1
  • @Phillipos, I'm the OP. I made one correction, and one clarification in the problem definition. Correction: You're right; the file delimiters are /, not /. (I work both on a Windows machine and a Mac, so I get these mixed up.) Clarification: The number of / characters will vary from file to file, so -k 4 will not always work.
    – JoshG
    Apr 22 at 11:02
0

Using any awk in any shell on every Unix box no matter which characters your paths contain (other than newlines):

$ cat tst.awk
BEGIN { FS=OFS="\t" }
{
    file = $0
    sub(".*/","",file)
    paths[file] = (file in paths ? paths[file] OFS : "") $0
}
END {
    for ( file in paths ) {
        print paths[file]
    }
}

$ awk -f tst.awk file
dir1/sub-dir1/.../filename2 author date dir2/sub-dir2/.../filename2 author date
.       .       .
dir1/sub-dir1/.../filename1 author date dir2/sub-di2/.../filename1 author date

If the output order actually matters it's a trivial tweak - just tell us what the output order should be (e.g. first read, alphabetical by file name, alphabetical by directory, something else).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.