I would like to figure out how to manage my storage, which has been quite large. The idea would be to see unnecessary files, and archive/delete.
At the moment, I am simply doing `du -h max-depth 1 and piping the outputs into text files.
However, I would like more information in order to figure out how to archive/delete my current storage.
---What is the best way I could get the file type and size for all files, recursively? This would be a quick way to check for intermediate files which are useless
---I was thinking about outputting all file metadata (i.e. data user, file type, file size, timestamp) into one large tsv file, so I could subset this in Python/R based on size and file type. Is this possible? How would we go about getting this information?
---Are there methods to look at storage versus time?
---Are there any other diagnostics/fancy visualizations which could help me figure out how to organize/archive/delete the data currently stored?