Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Say I have below log files,

AA_XX_20130719185428.log
exec_xxx_cpb_20130712182453.log
13122013121327_AR_INC_DOBC_1.dprf.log
24122013_masterscript.LOG

Date part contains both date and time.

I want to delete all the 2 months back files.

I don't want to delete the file on the basis of last modified date but on the basis of the date included in the filename. The dates can be in any of these formats:

  • YYYYmmddHHMMSS
  • ddmmYYYYHHMMSS
  • ddmmYYYY.

How I can delete the files using shell script?

share|improve this question
1  
what is the date in 3rd file? 2013-12-20? Are date & time always seperated with '_'? –  Zelda Dec 30 '13 at 8:54
    
File 1 and 2 has YYYYmmddHHMMSS, file 3 ddmmYYYYHHMMSS file 4 ddmmYYYY. Any other formats used (mmddYYYYHHMMSS)? –  Zelda Dec 30 '13 at 10:19
    
No. I am using only these formats –  user55551 Dec 30 '13 at 11:44
    
Hi Zelda.. Above given file names are sample file name and I want to delete all 2 months back files from log folder. –  user55551 Dec 30 '13 at 11:49
    
Than my solution should work, but you can comment out last line to test #os.remove() –  Zelda Dec 30 '13 at 12:07
show 1 more comment

3 Answers

You can use the script following. Call with file-names to test and delete as arguments.

#!/usr/bin/env python
# encoding: utf-8

import os
import sys
import datetime

test_file_names = [x for x in """\
AA_XX_20130719185428.log
exec_xxx_cpb_20130712182453.log
13122013121327_AR_INC_DOBC_1.dprf.log
24122013_masterscript.LOG
""".split('\n') if x.strip()]

# appr. 2 meses
two_months = datetime.date.today() - datetime.timedelta(days=61)

testing = len(sys.argv) < 2
if testing:
    file_names = test_file_names
else:
    file_names = sys.argv[1:]

allow_datetime = set([
    len('YYYYmmddHHMMSS'),
    len('ddmmYYYYHHMMSS'),
    len('ddmmYYYY'),
])

for file_name in test_file_names:
    for part in os.path.splitext(os.path.basename(file_name))[0].split('_'):
        if len(part) not in allow_datetime:
            continue
        for ch in part:
            if not ch.isdigit():
                break
        else:
            dt = part
            if dt[4:6] == '20':  # ddmmYYYY
                yy = int(dt[4:8])
                mm = int(dt[2:4])
                dd = int(dt[:2])
            else:  # YYYYmmdd
                yy = int(dt[:4])
                mm = int(dt[4:6])
                dd = int(dt[6:8])
            try:
                d = datetime.date(yy, mm, dd)
            except ValueError:
                print 'wrong date', yy, mm, dd
                raise
            if testing:
                print '{:<6s} {:<40s} {}'.format(
                    'remove' if d < two_months else ' ', file_name, repr(d))
            elif d < two_months:
                print 'removing', file_name
                os.remove(file_name)
share|improve this answer
add comment

If you are using GNU date, you can take advantage of it's date manipulation abilities:

#!/usr/bin/env bash

## Define the age limit
lim=$(date -d "2 months ago" +%s); 

## Find all files in the current directory and
## sub directories
find . -type f | 

## Extract the longest string of numbers (we assume that is the date)
## and change mmdd to ddmm
perl -lne '
    ## skip file names that don't have enough numbers 
    /(\d{1,8})/ || next; $d=$1;
    next unless $d;
    ## Change YYYYmmddHHMMSS to mm/dd/YYY 
    if($d=~/201\d$/){$d=~s|(\d{2})(\d{2})(\d{4})|$2/$1/$3|}

    ## Change  ddmmYYYYHHMMSS or ddmmYYYY to mm/dd/YYY 
    else{$d=~s|(\d{4})(\d{2})(\d{2})|$2/$3/$1|}; 

    ## Print the original file name and the modified date string
    print "$_ $d" if 
' | 

## Read the file name into $f and the date into $d
while read f d; do 
## If this date is older than $lim, delete the file
 if [ "$(date -d "$d" +%s)" -lt "$lim" ]; then 
   rm "$f"; 
 fi
done

This can be condensed into a "one-liner" that you can copy/paste directly into the terminal:

lim=$(date -d "2 months ago" +%s); 
find . -type f | perl -lne '
 /(\d{1,8})/ || next; $d=$1; 
 if($d=~/201\d$/){$d=~s|(\d{2})(\d{2})(\d{4})|$2/$1/$3|}
 else{$d=~s|(\d{4})(\d{2})(\d{2})|$2/$3/$1|}; print "$_ $d"' | 
while read f d; do [ "$(date -d "$d" +%s)" -lt "$lim" ] && echo rm "$f";   done

CAVEATS: This solution assumes that your file names contain no spaces and that the longest string of digits in the file name will always be the date.

share|improve this answer
add comment
find . -type f -mtime +60 -exec rm -f {} \;

The -mtime is using file modified time

find . -type f -atime +60 -exec rm -f {} \;

The -atime uses access time

also with ls -lc you can see for inode creation time. This changes when inode data is updated as in using chmod, chown or chgrp for example.

share|improve this answer
    
I will advice, NOT to use the -f flag while deleting the search output. You can use -i flag to ask interactively for deletion comfirmation –  SHW Dec 30 '13 at 8:49
3  
first code deletes files based on the modification date, which is exactly not what OP asked for –  Zelda Dec 30 '13 at 8:52
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.