And the mandatory sed
solution:
sed '1s/^/#/;x;G;\_#\([^#]*\)#.*\n\1/_s/\n.*//;s/\n\(.*\)/\1#/;h;$! d;x;s/^#//;s/#$//;y/#/\n/'
The script collects paths in the hold space. For each new line, the hold space gets appended to the pattern space to check, whether it already occured.
This solutions assumes that the character #
is not used in the file. Otherwise use a different character or, if you use GNU sed
, use the short version at the bottom of the post.
Explanation in detail:
1s/^/#/
For portability, the #
character is used to separate the paths in the hold space. For the first line we need to start with an initial #
x;G
By exchanging the spaces and appending the hold space, we have the list of already occured buffers first, then the new path.
\_#\([^#]*\)#.*\n\1/_s/\n.*//
If the \_..._
address matches, the new path is a subpath of an earlier path, so remove it.
s/\n\(.*\)/\1#/
There is still a newline in our space, so the path is new and we add it to the list.
h;$! d
Save the new list to the hold space and start over, if this was not the last line.
x;s/^#//;s/#$//;y/#/\n/
For the last line, remove the #
at beginning and end and replace the other #
by newlines.
Alternative for GNU sed
This can be done more compact with GNU extensions on sed
, if you don't mind if the order gets reverted:
sed 'G;\_^\([^\n]*\)/.*\n\1\n_s/[^\n]*\n//;h;$! d;x;s/^\n//;s/\n$//'
Explanation as above, but using the newlines as separators instead of adding #
.
/fff/ggg/hhh/iii
"redundant"?sort -t'/' -u -k1,2
will work for given sample... it uses first two columns as comparison to find unique entries and retain only first entry