3

Context

I have backup scripts running which create tar.gz files and transfer them to a dropbox folder on my headless linux VM.

Problem

I do not know how to create a script which will detect the oldest file among files named according to the scheme /root/Dropbox/apache2-backup-%Y-%m-%d-%H-%M.

I then want to move this file to a /root/Dropbox-archive/ folder, because there is only 2GB of room on Dropbox.

  • 1
    If you just ls -1 /Dropbox/apache2-backup-*, doesn't it automatically order alphabetically by name, i.e. by date (since date is formatted with most significant digit on the left)? Then you can get the oldest with |head -1 (assuming there are no newlines in file names). – Sparhawk Oct 24 '15 at 0:22
  • @Sparhawk thanks! I can then just mv "$(ls -1 /root/Dropbox/apache2-backup-* | head -1)" /root/Dropbox-archive/ ? – Alex Stewart Oct 24 '15 at 0:32
  • 1
    Yes, exactly. I'll write an answer with more details. – Sparhawk Oct 24 '15 at 1:01
2

"Simple" solution

ls -1 /Dropbox/apache2-backup-* should order alphabetically by name, i.e. by date (since date is formatted with most significant digit on the left). You can then get the oldest by piping to head -1 (assuming there are no newlines in file names). Generally, piping from ls is a bad idea, but it should be fine in this case since you know that there are no special characters in the file names.

mv "$(ls -1 /Dropbox/apache2-backup-* | head -1)" /root/Dropbox-archive/

I'd probably also recommend using mv --no-clobber (or mv -n) instead, so you don't accidentally overwrite files.

"Proper" solution

If you are really worried about special characters, you can use the following instead.

to_move="$(find /Dropbox -maxdepth 1 -type f -name 'apache2-backup-*' -print0 | sort -z | tr '\0\n' '\n\0' | head -1 | tr '\0\n' '\n\0')"
mv "$to_move" /root/Dropbox-archive/

Explanation

  • find /Dropbox -maxdepth 1 -type f -name 'apache2-backup-*' -print0: return normal files (-type f) in /Dropbox without descending into subdirectories (-maxdepth 1), that match the pattern -name 'apache2-backup-*'. Delimit by null character, instead of newlines -print0.
  • | sort -z: sort based on null character (N.B. not all sorts can do this).
  • | tr '\0\n' '\n\0': swap null and newlines for processing by head.
  • | head -1: return first line.
  • | tr '\0\n' '\n\0': swap null and newlines back.
  • mv "$to_move" /root/Dropbox-archive/: do the move!

Or, in a single line:

mv "$(find /Dropbox -maxdepth 1 -type f -name 'apache2-backup-*' -print0 | sort -z | tr '\0\n' '\n\0' | head -1 | tr '\0\n' '\n\0')" /root/Dropbox-archive/
  • In the title it is files, plural. – Arthur2e5 Oct 24 '15 at 16:32
  • @Arthur2e5 Did you downvote for that?? In the question body, it is singular, twice. Both these scripts can be easily modified for multiple files, by replacing head -1 with head -n, where n is the number of files to move. – Sparhawk Oct 24 '15 at 21:36
  • @Sparhawk +1 for a great answer and explaining each part of it. I do wonder if there is a way to simply find "the oldest file" recursively in a subdirectory regardless of name, but that is a different question I suppose. – Wildcard Oct 24 '15 at 22:24
  • 1
    @Wildcard Thank you! (And I've got another downvote for some reason!) Yes, the problem with "oldest file" is that it's a bit ambiguous. Probably the best way to do it is with crtime, but that's quite difficult to obtain. You could certainly write a script to extract and sort by it, but it'd be a fair bit trickier than this. – Sparhawk Oct 24 '15 at 23:45
  • Well, since the question goes like that…… I will cancel my downvote and correct the title to match the question content. – Arthur2e5 Oct 25 '15 at 7:40
2

If you are only moving the oldest one, use this:

_mv(){ mv -- "$1" /root/Dropbox-archive/; }
_mv /Dropbox/apache2-backup-*

But your title doesn't say so.


Since processing ls output is a bad idea, here is a better approach:

if [ "$BASH" ]; then
# If you happen to use bash or any other shell that has some similar array:
move_first_things(){ mv -- "${@:0:50}" /root/Dropbox-archive/; }
else # Unfortunate POSIX way
# As long as copy-pasting is acceptable, loop-unrolling with
# `mv -- "$1" "$2" "$3"... /root/Dropbox-archive/` is better since it calls `mv` less.
move_first_things(){
    local max=50 count=0
    while [ "$count" -lt "$max" ]; do
        mv -- "$1" /root/Dropbox-archive/
        shift
        : $((count = count + 1))
    done
}
fi

And since globbing sorts the output just like ls does:

move_first_things /Dropbox/apache2-backup-*

PS: This can be indeed done with external programs (mikeserv, explanation here). With a bit more care and extra obfuscation, people can have it done safely. But I am still going to use the nice filename in this case.

PPS: don_crisst mentioned some zshism. zsh has nice support for anonymous things, like functions, param expansions and arrays. Still using the filename way (since bash doesn't have such sorting operators), the whole thing can be written as baks=( /Dropbox/apache2-backup-* ); mv "$baks" /root/Dropbox-archive/ (referencing an array like this simply gives its first member), or for multiple things, mv "${baks[@]:0:50}" /root/Dropbox-archive/. Well, this looks a bit better than those wrapper functions actually.

  • 1
    don't really understand your code or how it works. – Alex Stewart Oct 24 '15 at 0:55
  • @AlexStewart The number of files to move can be controlled by a function that only moves some of its first arguments. In the _mv case, I only moved "$1", the first argument. Given that globs are sorted, it is the first word from the globbed list and therefore the oldest file. – Arthur2e5 Oct 24 '15 at 0:58
  • @AlexStewart The POSIX one with shift is similar, if you know that shift shifts the second arg to the first, the third arg to the second and vice versa. The bash one takes a trick in the parameter expansion of arrays. – Arthur2e5 Oct 24 '15 at 1:01
  • globbing doesn't sort the output just like ls. ls sorts according to the currently defined $LC_COLLATE locale configuration as the shell does, yes, but it also accepts many arguments -[RfLCmtr] that affect its sort order besides - and it treats symbolic links differently to boot. The bash trick - how do you come up with 50? – mikeserv Oct 26 '15 at 7:41
  • @mikeserv Our scope is only inside numbers, and I remember per ANSI C numbers 0-9 are always continuous, so locales don't matter. 50 is only used as an example. – Arthur2e5 Oct 26 '15 at 12:38
1
(   IFS=/                                      # split on /
    set -ef /Dropbox/apache2-backup*           # set arg array to last glob
    for    f in    $(\ls -rtd "$@";echo /)     # iterate over sorted/split array
    do     [ -z "${f##D*}" ]  ||               # ignore dirname
           mv "${1%/*}/${f%?}" \
              "/root${1%/*}-archive"           # do sorted mv
    done                                       # do && break to do only one mv
)

For whatever reason people seem to think that you can't reliably sort a list of arguments with ls. I think this is because these same people are trying to delimit pathnames with newlines. But the thing is, pathnames do not delimit on newlines - they never have. Pathnames delimit on / - that is the pathname delimiter, and so newlines only confuse things. If you want to reliably parse ls output, you've got to split it correctly.

  • @Arthur2e5 - set -- is not an idiom, it is a statement that means clear all positional parameters. doing set positional and set -- positional are the same thing - and the -- means nothing at all in the second case. doing set -- $var and set $var, however, are very different - if $var is empty or is set to [-+].* in the first case set will just clear the arg array, in the second if it is empty it will write a list of all declared variables, or if it is [-+].* it will attempt to set a shell option. And so set -- is useful in some cases, and in others it is not. – mikeserv Oct 26 '15 at 21:11
  • 1
    Sorry for being unclear, but I meant to say 'something with a fixed pattern that makes people feel familiar about what it does'. – Arthur2e5 Oct 27 '15 at 1:55
  • @Arthur2e5 set should be familiar enough to any basic shell user, i would think. it is one of the oldest and most fundamental commands in unix shells. – mikeserv Oct 27 '15 at 1:59
  • @Arthur2e5 --- yo. – mikeserv Feb 22 at 6:59
1

The question specified oldest file. ls -rt lists the files sorted by modification time, oldest first. This answer does not depend on the filename format except for the prefix identifying a backup file. The date part of the filename is not considered here:

mv "$(ls -tr /root/Dropbox/apache2-backup-*| head -1)" /root/Dropbox-archive/

Sorting by file time is not better than sorting by filename and vice versa. It's largely personal preference and depends on the situation and culture.

In both of these cases, and most, it's safer to enclose the command expansion in double quotes " like this "$( ... )", which will handle all special characters in the backup file filenames (spaces, punctuation, tabs, etc) except for filenames containing newlines.

  • N.B. this is sorting by modification date. I'm not sure if these files may have been modified. Perhaps it'd be better to sort by ctime or crtime? crtime is non-trivial, and I'd presume that the hardcoded name would be more reliable. – Sparhawk Oct 24 '15 at 21:45
  • Sure. This example just shows an alternative that doesn't rely on the filename. – RobertL Oct 24 '15 at 22:47
  • 1
    That's valid, just making sure we know what "oldest" means in this case. (Also, do you know what the 1% of special char problems are? Also also, if you want to give the absolute path for /bin/ls, why not for mv and head too?) – Sparhawk Oct 24 '15 at 23:47
  • Heh, you could take it to mean what was said in the first line of the answer. In my experience, the only "special char" that poses a threat to this construct would be embedded newlines, which in my experience is more like .001%. What is it in your experience? Yea, I take back /bin/ls, not such a great idea. Just left over brain cruft from another question. :-) I like your Simple Solution best, except that it was kind of obfuscated by your Proper Solution, which, though well documented, IMHO is impractical in respect to the original post and for day to day maintenance. – RobertL Oct 25 '15 at 21:01
  • Thank you. Sorry, I still don't quite understand. I thought "$(...)" should work for all special cases? I just tested, and it's fine for files with newlines. However, your answer suggests it doesn't work for all special cases? Hence, what are the special cases that I fails for? – Sparhawk Oct 25 '15 at 22:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.