An Oceanographer friend at work needs to back up many months worth of data. She is overwhelmed so I volunteered to do it. There are hundreds of directories to be backed up and we want to tar/bzip them into files with the same name as the directory. I can do this easy enough serially - but - I wanted to take advantage of the several hundred cores on my work station.
Question: using find with the
-P args or GNU Parallel, how do I tar/bzip the directories, using as many cores as possible while naming the end product:
I have used find to bunzip 100 files simultaneously and it was VERY fast - so this is the way to approach the problem though I do not know how to get each filename to be that of each directory.