I'm writing something for backup. And I'm working with really Big files/folders (1 million++, 100TB++)
For improve my speed, at least I need to pick 10 biggest folders in tree and for that, I need to know some information in tree.
- What is 10-20 biggest folders in sub-tree? (disk usage)
- Which folders has most partial files. (Usage doesn't matter, its for 4k random speed)
I can't get the information with
du -kh because its taking too long.
Now I'm thinking; creating folder tree first, after that I should calculate in my local storage without files, than I'm ready to go.
BUT! With this way, I just know folder tree. I still don't know which folder has biggest and most particulated files.
I couldn't any way to find both information and now I'm thinking "guessing method" with option 2.
What should I do?