So I have .lzo
files in /test01/primary
folder which I need to uncompress and then delete all the .lzo
files. Same thing I need to do in /test02/secondary
folder as well. I will have around 150 .lzo
files in both folders so total around 300 .lzo
files.
From a command line I was running like this to uncomressed one file lzop -d file_name.lzo
.
What is the fastest way to uncompressed all .lzo
files and then delete all .lzo
files from both folders simultaneously. Below is the code I have:
#!/bin/bash
set -e
export PRIMARY=/test01/primary
export SECONDARY=/test02/secondary
parallel lzop -dU -- ::: {"$PRIMARY","$SECONDARY"}/*.lzo
I want to uncompress and delete .lzo
files parallelly in both PRIMARY and SECONDARY folder simultaneously. With my above code, it does in PRIMARY first and then in SECONDARY folder. How can I achieve parallellism both in PRIMARY and SECONDARY simultaneously?
Also does it uncompress all the files and then delete later on or uncompress one file and then delete that file and then move to next one?
I tried with this but it doesn't work. It just works on first 40 files and after that it doesn't work at all.
#!/bin/bash
set -e
export PRIMARY=/test01/primary
export SECONDARY=/test02/secondary
parallel -j 40 lzop -dU -- ::: "$PRIMARY"/*.lzo &
parallel -j 40 lzop -dU -- ::: "$SECONDARY"/*.lzo &
wait
p_script
with two lines:parallel lzop -dU -- ::: path1/*.lzo
andparallel lzop -dU -- ::: path2/*.lzo
make it executable and then runparallel ::: ./p_script
(you should further optimize with-j
...)