4

I have directory that has a number of sub-directories and would like to find any duplicates. The folder structure looks something like this:

└── Top_Dir
    └── Level_1_Dir
        ├── standard_cat
        │   ├── files.txt
        ├── standard_dog
        │   └── files.txt
        └── standard_snake
            └── files.txt
    └── Level_2_Dir
        ├── standard_moon
        │   ├── files.txt
        ├── standard_sun
        │   └── files.txt
        └── standard_cat
            └── files.txt
    └── Level_3_Dir
        ├── standard_man
        │   ├── files.txt
        ├── standard_woman
        │   └── files.txt
        └── standard_moon
            └── files.txt

With the above example I would like to see an output of:

/top_dir/Level_1_Dir/standard_cat
/top_dir/Level_2_Dir/standard_cat
/top_dir/Level_2_Dir/standard_moon
/top_dir/Level_3_Dir/standard_moon

I have been doing some searching on how to get this done via bash and I got nothing. Anyone know a way to do this?

1

This worked using bash on Ubuntu. It only matches duplicate directories irrespective of depth in the tree. The portion within the $() builds a list of pipe-separated directory names by counting duplicates in the last column of ls -l. This pipe-separated list is filtered using grep over the list of all directories. Also, not accounting for other files (didn't use whole word matches etc.)

> ls -lR Top_Dir/ | grep -E $(ls -lR Top_Dir/ | grep ^d | rev | cut -d" " -f1 | rev | sort | uniq -d | head -c -1 | tr '\n' '|') | grep -v ^d | sed 's/://'

Top_Dir/Level_1_Dir/standard_cat

Top_Dir/Level_2_Dir/standard_cat

Top_Dir/Level_2_Dir/standard_moon

Top_Dir/Level_3_Dir/standard_moon

  • First, thanks for your reply. When running the command I am getting a head: illegal byte count -- -1 error. Any thoughts on that? I have been paying with the head command but not seeing what is wrong with it. – dino Jun 10 '16 at 0:52
  • It's probably specific to your version of head. The link also contains other solutions to truncate the last line. – Jedi Jun 10 '16 at 0:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.