I am looking to create a script - a linux or a python scrip so that it can create the following things for me as well as Verify the files for me which are copied from a folder.

I Have two folders:

FolderA has 300 .xls files - This folder is missing some files which are in folder B currently.

FolderB has 500 .xls files

I want to copy select few 100 files from Folder B to folder A. Then want the script to verify that all the files currently residing now in folder A(should be 400 now after copying 100 files from B) also exist in folder B.

Then I want the script to zip all these files separately as its own bzip2 file. Basically there will be 400 bzip2 files(one for each excel) in the end when the process is completed.


How will the script know to automatically differentiate what files to compress in this folder? is there a specific naming convention? If there is not, you may need to merge the files you want to compress from Folder A and Folder B to Folder C. The following script will compress each file. If you don't want to keep the original .xls files remove the -k

for f in $files; do
    bzip2 -k $f
  • Thats what I want the script to do, I want to see if my script can pick up listoffiles.txt or something similar where I would have listed the files which need to be compressed to bzip2. Can you tell me a workaround using that?
    – mywayz
    Jul 19 '19 at 14:14

It is unclear to me how you want the 100 missing files to be selected. This picks 100 at random (if you want to simply select the first 100 missing files, remove --shuf):

# copy 100 files from folderB
parallel --shuf -j1 --halt now,success=100 [ ! -f $a/{/} ] '&&' cp {} $a/{/} ::: $b/*
# Verify all files in folderA is in folderB
parallel [ ! -f $b/{/} ] '&&' echo {} is in $a but not in $b ::: $a/*
# bzip2 them all
parallel bzip2 ::: $a/*

GNU Parallel is used primarily so that you can run a bzip2 per CPU thread.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.