I am working on setting up cron jobs,and I am new to this. The purpose is to analyze the new data that is generated.The data is present in directories.
I want to sync only the latest data/that has not been analyzed for which I created a list of directories that have been analyzed(to be excluded).The below command I am using to sync data from source with a certain pattern to destination.
rsync -av --exclude-from=/home/list1 /datasets001/*/Pr*/*RNA* $TMPDIR/raw_fastq
The issue is the above command behaves same as the below command and does not exclude any data mentioned in list1.
rsync -av /datasets001/*/Pr*/*RNA* $TMPDIR/raw_fastq
few lines of list1 are present below:
Also,any other suggestions to sync the new data and analyze it(based on date) are welcome.