I was wondering if there is a way to grab the oldest 100 files from a directory that has a 1000 or more in it with the same prefix (cdr12345.csz, cdr32342.csv, cdr 98876.csv, etc..) and list the names of those 100 files out to file.txt, that I can cat out and read later, and then zip those 100 files up. I can only do 100 at a time, so how can I just get the oldest 100 first? Caveat is that the zip file must be under 500 mgb.
In terms of identifying the oldest 100 files, you can do something like:
ls -1tr /path/to/directory/cdr* | head -n100 > filelist.txt
-1 will list one file per line (not strictly needed here), the
-t will sort by modification time, newest first, and the
-r will reverse the sort order (i.e., will make it list newest last). Pipe that output to
head -n100 will list the first 100 lines (i.e., the 100 oldest files). I then redirect the output of that to
filelist.txt -- that file contains the names of the 100 oldest files.
You could then zip those files with:
zip backup -@ < filelist.txt
That will create a file called
backup.zip with the content of the files in the
-@ option instructs
zip to read the file names from standard input, and the redirection provides that list from the file.
Once you've tested that much and see if it does what you want, you could then remove the files in
xargs rm < filelist.txt
That'll leave you with:
filelist.txtthe list of the oldest files
backup.zipthe content of the oldest files
Note that this doesn't address your 500MB file size limit. One approach would be to examine the zip file's size after you create it, and if it's too big, decrease the number of files you include in the archive.
While saving and re-parsing filenames can be tricky, if your filenames don't contain spaces or newlines, you can deal with them less carefully. I'll show a method using zsh where the filenames are never parsed directly.
Gather the filenames into an array:
files=( cdr*.csv(om[-100,-1]) )
This says to match filenames that start with
cdr and end with
.csv, but to
order those files by their modification times and then select only the 100 oldest of those.
You can the print those to a file in various ways, such as:
print -l $files > filelist.txt printf 'File: %s\n' $files
To create a zipfile of those files, then:
zip zipfile $files
... where in each instance, zsh replaces the
$files variable with each element. A more careful way of doing the same expansion would be to use
ls -1tr | head -n 100
will give you a list of the 100 oldest files in a directory.
You can save that output to a file (e.g. zip.in). You can then use that file to create a zip file. For example:
zip zipfile.zip $(cat zip.in)
In terms of the zip file having to be under 500MB, you can calculate the total size of the individual files to see if they exceed 500MB. However, since zip is doing some compression though, this is not accurate. You will only know once the zip file is created.
You could add each file in the zip.in list one at at time to zipfile.zip. After adding each one, you could then check the size of zipfile.zip to see if it exceeds 500MB.