0

I was wondering if there is a way to grab the oldest 100 files from a directory that has a 1000 or more in it with the same prefix (cdr12345.csz, cdr32342.csv, cdr 98876.csv, etc..) and list the names of those 100 files out to file.txt, that I can cat out and read later, and then zip those 100 files up. I can only do 100 at a time, so how can I just get the oldest 100 first? Caveat is that the zip file must be under 500 mgb.

2

In terms of identifying the oldest 100 files, you can do something like:

ls -1tr /path/to/directory/cdr* | head -n100 > filelist.txt

The -1 will list one file per line (not strictly needed here), the -t will sort by modification time, newest first, and the -r will reverse the sort order (i.e., will make it list newest last). Pipe that output to head -n100 will list the first 100 lines (i.e., the 100 oldest files). I then redirect the output of that to filelist.txt -- that file contains the names of the 100 oldest files.

You could then zip those files with:

zip backup -@ < filelist.txt

That will create a file called backup.zip with the content of the files in the filelist.txt. The -@ option instructs zip to read the file names from standard input, and the redirection provides that list from the file.

Once you've tested that much and see if it does what you want, you could then remove the files in filelist.txt:

xargs rm < filelist.txt

That'll leave you with:

  1. filelist.txt the list of the oldest files
  2. backup.zip the content of the oldest files

Note that this doesn't address your 500MB file size limit. One approach would be to examine the zip file's size after you create it, and if it's too big, decrease the number of files you include in the archive.

4
  • Thank you, Gents! This is great! Is there anyway to pass a date value to make sure I am getting the current date? I may need to do this but don't see it as a requirement, YET...
    – Scott Mick
    Oct 5 '18 at 16:16
  • Also, the -@ does not jive with gzip.
    – Scott Mick
    Oct 5 '18 at 16:18
  • @ScottMick gzip won't take a collection of files and produce a single file, so you'll end up with 100 separate files. You could use plain ole zip (like you said in your question), or you could use tar in conjunction with gzip. Oct 6 '18 at 2:23
  • Thank you! The modification I made was to use ls -1t since the r option was doing the opposite of what is expected (MacOS) Nov 8 '19 at 17:26
0

While saving and re-parsing filenames can be tricky, if your filenames don't contain spaces or newlines, you can deal with them less carefully. I'll show a method using zsh where the filenames are never parsed directly.

Gather the filenames into an array:

files=( cdr*.csv(om[-100,-1]) )

This says to match filenames that start with cdr and end with .csv, but to order those files by their modification times and then select only the 100 oldest of those.

You can the print those to a file in various ways, such as:

print -l $files > filelist.txt
printf 'File: %s\n' $files

To create a zipfile of those files, then:

zip zipfile $files

... where in each instance, zsh replaces the $files variable with each element. A more careful way of doing the same expansion would be to use ${files[@]} instead.

0

The command

ls -1tr | head -n 100

will give you a list of the 100 oldest files in a directory.

You can save that output to a file (e.g. zip.in). You can then use that file to create a zip file. For example:

zip zipfile.zip $(cat zip.in)

In terms of the zip file having to be under 500MB, you can calculate the total size of the individual files to see if they exceed 500MB. However, since zip is doing some compression though, this is not accurate. You will only know once the zip file is created.

You could add each file in the zip.in list one at at time to zipfile.zip. After adding each one, you could then check the size of zipfile.zip to see if it exceeds 500MB.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.