0

The following syntax will remove the files under hive folder:

/usr/bin/find /var/log/hive -type f -print -delete

I am trying to do the following:

Remove the oldest files under /var/log/hive only if folder size is more than 10G

NOTE - the deletion process will stop when size under hive folder is exactly 10G , so purging process will start if size is more then 10G

Can we create this solution with find command or maybe another approach?

  • oldest in the sense?? can you specify number of days? – msp9011 Aug 9 '18 at 12:25
  • I mean the most oldest files , as ls -ltr will show the date and time of the files , and the removing will be from the oldest files according to ls -ltr info ( but I mentioned the ls -ltr only for example ) – yael Aug 9 '18 at 12:34
  • folder or file ?? – msp9011 Aug 9 '18 at 12:35
  • its like you do du -sh . (under hive folder ) , if size is more then 10G then need to triger the deletion files – yael Aug 9 '18 at 12:37
  • need to delete only files – yael Aug 9 '18 at 12:58
4

On a GNU system, you could do something like:

cd /var/log/hive &&
  find . -type f -printf '%T@ %b :%p\0' |
    sort -zrn |
    gawk -v RS='\0' -v ORS='\0' '
      BEGIN {max = 10 * 1024 * 1024 * 1024} # 10GiB; use max=10e9 for 10GB
      {du += 512 * $2}
      du > max {
        sub("[^:]*:", ""); print
      }' | xargs -r0 echo rm -f

That is sort the regular files by last modification time (from newest to oldest), then count their cumulative disk usage (here assuming there are no hard links) and delete every file when we've passed the 10GiB threshold.

Note that it doesn't take into account the size of the directory files themselves. It only considers the disk usage of regular files.

Remove echo when satisfied with the result.

On one line:

find . -type f -printf '%T@ %b :%p\0' |sort -zrn|gawk -vRS='\0' -vORS='\0' '{du+=512*$2};du>10*(2^30){sub("[^:]*:","");print}'|xargs -r0 echo rm -f

To delete only *.wsp files when the cumulative disk usage of all regular files goes over 10GiB, you'd want to list the non-wsp files first. And at the same time, we can also account for the disk usage of directories and other non-regular files we were missing earlier:

cd /var/log/hive &&
  find . \( -type f -name '*.wsp' -printf WSP -o -printf OTHER \) \
     -printf ' %T@ %b :%p\0' |
    sort -zk 1,1 -k2,2rn |
    gawk -v RS='\0' -v ORS='\0' '
      BEGIN {max = 10 * 1024 * 1024 * 1024} # 10 GiB
      {du += 512 * $3}
      du > max && $1 == "WSP" {
        sub("[^:]*:", ""); print
      }' | xargs -r0 echo rm -f
  • can I run it in one line ? – yael Aug 9 '18 at 12:59
  • @yael, Sure, but why would you want to do that? All shells accept commands on several lines (though it can get awkwards with csh or tcsh). – Stéphane Chazelas Aug 9 '18 at 13:01
  • because I want to run it crom crontab -:) – yael Aug 9 '18 at 13:05
  • @yael, OK. Though here, I'd rather create a script with that code and have the crontab run that script. See edit for the one line version. You may want to test that it works OK first. I've not tested it myself. – Stéphane Chazelas Aug 9 '18 at 13:08
  • lets say that we want to delete only the files that are ended with ".wsp" , can you please give update to show the second approach that delete only the .wsp deletion – yael Aug 9 '18 at 13:23
1

Try this,

Option 1: To delete folder older than 90days and more than 10G

find /var/log/hive -size +10G -mtime +90 -type f -print -delete

Option 2: To delete the oldest folder

find /var/log/hive -size +10G -type f -printf '%T+ %p\n' | sort | head -n 1 | cut -d" " -f2 | xargs rm
  • so both options will delete the files until size will be exactly 10G ? ( in case size became for example 11.5G ) – yael Aug 9 '18 at 12:36
  • 3
    This deletes files larger than 10Gb. – Kusalananda Aug 9 '18 at 12:36
  • I not want to delete files that are more then 10G , what I mean is that --> in case of size under hive folder is more then 10G , only then the deletion mechanizem will work to delete the oldest files , – yael Aug 9 '18 at 12:39
1

How about

while test "$(du -s /var/log/hive | cut -f1)" -gt 10000000 ; do rm -i /var/log/hive/"$(ls -t /var/log/hive | tail -1)" ; done

?

  • 1
    Related: unix.stackexchange.com/questions/128985 – Kusalananda Aug 9 '18 at 12:37
  • dose it stop to delete when size is 10G ? ( what we need is to stop the deletion when size iz exactly 10G ) , I also update the question with note so it will be more clear – yael Aug 9 '18 at 12:41
  • @yael: Yes, it stops if the size reaches 10GB or less (10GB, not 10GiB). @Kusalananda: I took this from unix.stackexchange.com/questions/242496/…. How to select the oldest file in a directory without parsing ls? – Hermann Aug 9 '18 at 16:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.