I need to keep files 4 months from now (3 previous month + 1 current month) in directory and tar the rest. But the problem is my files consist of different format.


As the above example, the date is in either dd-mm-yyyy or yyyy-mm-dd format, i need to extract the date from filename and compare if it is not in the (3 previous months + 1 current month) range, then will tar it.

Today is 2nd of March, files from 1st of Dec 2015 until today will remain, but will tar the others.

Expected tar files from example:


I can write the regex for these 2 types date format, but i don't know how to combine them.

Regex for yyyy-mm-dd:

ls |grep -Eo '[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}'   

Regex for dd-mm-yyyy:

ls |grep -Eo '[[:digit:]]{2}-[[:digit:]]{2}-[[:digit:]]{4}'

Compare date:

files=($(ls | awk -v d=$(date -d '3 months ago' +%m-%Y.log) 'DATE_FROM_FILES< d {print;}' ))

tar cvzf archive.tar.gz "${files[@]}"

Assuming the regular filename format as given, where the date is the last part before the extension, and it's immediately preceded by an underscore, plus that the extension is always .txt, you could use a script like the following

d=$(date -d '3 months ago' +%Y-%m)
ls | while read f ; do
    df=$( date --date $df1 +%Y-%m 2>/dev/null || date --date $df2 +%Y-%m)
    [[ "$df" < "$d" ]] && files=( $files $f )
tar cvzf archive.tar.gz "${files[@]}"
# rm ${files[@]}

The key aspects here are: the date string to compare against must have the year value to the left of the month value in order for the alphabetical ordering to make sense as a date ordering. Next, the date command can take the date either as yyyy-mm-dd or as dd/mm/yyyy; thus, $df1 is obtained from the filename by first eliminating everything up to and including the last underscore in the filename, and then eliminating the .txt extension. $df2 is obtained from that by replacing - with /.

In that way the one or the other of $df1 and $df2 should work to determine the date part of the file name, and to revamp it into the yyyy-mm format for comparison. Note that date fails with an error message of for a date dd-mm-yyyy, which then the script pipes that to /dev/null

Then it's just a matter of collating the file names that are prior to the cut date, tar them up, and (commented out) remove the tarred files.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.