4

Consider a directory with MySQL backup files with timestamps as part of the filename:

-rw-rw-r-- 1 ubuntu ubuntu 35856184 Nov 16 16:00 db_2013-11-16_1600.sql
-rw-rw-r-- 1 ubuntu ubuntu 35856915 Nov 16 17:00 db_2013-11-16_1700.sql
-rw-rw-r-- 1 ubuntu ubuntu 35857565 Nov 16 18:00 db_2013-11-16_1800.sql
-rw-rw-r-- 1 ubuntu ubuntu 35858254 Nov 16 19:00 db_2013-11-16_1900.sql
-rw-rw-r-- 1 ubuntu ubuntu 35860276 Nov 16 20:00 db_2013-11-16_2000.sql
-rw-rw-r-- 1 ubuntu ubuntu 35861583 Nov 16 21:00 db_2013-11-16_2100.sql
-rw-rw-r-- 1 ubuntu ubuntu 35863630 Nov 16 22:00 db_2013-11-16_2200.sql
-rw-rw-r-- 1 ubuntu ubuntu 35864868 Nov 16 23:00 db_2013-11-16_2300.sql
-rw-rw-r-- 1 ubuntu ubuntu 35866095 Nov 17 00:00 db_2013-11-17_0000.sql
-rw-rw-r-- 1 ubuntu ubuntu 35887731 Nov 17 01:00 db_2013-11-17_0100.sql
-rw-rw-r-- 1 ubuntu ubuntu 35888871 Nov 17 02:00 db_2013-11-17_0200.sql
-rw-rw-r-- 1 ubuntu ubuntu 35888871 Nov 17 03:00 db_2013-11-17_0300.sql
-rw-rw-r-- 1 ubuntu ubuntu 35889319 Nov 17 04:00 db_2013-11-17_0400.sql

These actually go on since September 2012! I need to delete all backups except for the last backup of each month. That is, these files should be left:

db_2012-09-30_2300.sql
db_2012-10-31_2300.sql
db_2012-11-30_2300.sql
db_2012-12-31_2300.sql
db_2013-01-31_2300.sql
db_2013-02-28_2300.sql
db_2013-03-31_2300.sql
db_2013-04-30_2300.sql
db_2013-05-30_2300.sql
db_2013-06-30_2300.sql
db_2013-07-31_2300.sql
db_2013-08-31_2300.sql
db_2013-09-30_2300.sql
db_2013-10-31_2300.sql
db_2013-11-20_0700.sql # Because this month has not finished yet!

I could write a Bash/Python script to create lists of each month, remove the last item from the list and then delete one-by-one the remaining files. Alternatively, the script could move the last files from each month to a temp directory, remove everything, then put the files back.

However I wonder if there is some way to simply tell rm (or rm with find and awk and sort) to ignore the last file of the month. Is there such a magic spell?

I do recognise that life would be easier if I could just save the first file from each month (which would be only a 1 hour difference from saving the last file from each month) but that is not acceptable to others in the organization who fail to see that this essentially provides the same protection.

  • sort -r | uniq -w11 gets you a list of the files you want to keep. – ctrl-alt-delor Nov 20 '13 at 12:13
2

With the file zz containing the list of file names, this works, so just replace cat zz.

cat zz | grep -vF -f <(cat zz|sort -r|uniq -w11)

e.g. echo *.sql | grep -vF -f <( echo *.sql | sort -r | uniq -w11 ) | xargs rm

As is, it won't work if spaces in file names, and very fragile to filename length.

  • Thank you Richard. Is the goal to show the names of the files to keep? On my Ubuntu 12.10 test system it is showing all the files in zz, just like cat zz alone would do. I do see where you are going with sort and uniq though, and that should give me some headway to run with. Thanks. – dotancohen Nov 20 '13 at 12:56
  • I'm not there yet, but this tip gets me a list of the files to keep and I can work from there. Thanks! – dotancohen Nov 20 '13 at 13:02
  • The <(…) is to get a list of files to keep. grep -v removes these names. – ctrl-alt-delor Nov 20 '13 at 13:08
1

A quick ugly brute force attempt in a script (could be done in one line, but need two for loops so it's kind of ugly in one line)

#! /bin/bash

for year in {2012..2013}
do
   for month in {01..12}
   do
      ls db_"$year"-"$month"*.sql 2>/dev/null | sort -r | tail -n +2 | xargs rm -f
   done
done

This of course assumes no special characters/whitespace in the file names, which seems to be your case.

  • Nice, thanks. I like the Bash loop, though I was hoping for something that would automatically figure that bit out. – dotancohen Nov 20 '13 at 16:10
1

Using ricchard's idea removing all except last in month:

rm -f `ls *.sql | sort | uniq w11 -D -u`

To view what will be deleted:

ls *.sql | sort | uniq w11 -D -u
-3

I would use find for this,

something along the lines of find "path" -type f -mtime +30 -exec rm {} \;

that -mtime +30 signifies 30 days but that can be changed easily to suite your needs.

edit alternatively you could set up a cron task with the above script instead use mv to move it to a convenient location.

  • 1
    Perhaps I failed to convey the idea. I need the last file for each month, not all files in the last 30 days. – dotancohen Nov 20 '13 at 9:02
  • @dotancohen I do apologize, you can specify a time range with find but that starts to become rather cumbersome as you would need to start with -x weeks until now then -1month until -x weeks, something like <code>find /directory -type f -mtime -10 -mtime +5 </code> – daark Nov 20 '13 at 9:17
  • I am not looking for relative dates. Nor is this an operation that will need to be run periodically (cron). Thank you for trying, though. – dotancohen Nov 20 '13 at 9:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.