0

I want to copy the last used (or maybe created) files of a total size to another folder. Is this possible without additional tools?

I have an USB drive of a certain size that is less than the total size of a folder. As I can't copy all files to USB I like to copy based on latest usage until there is no more space. Ideally the method also supports updating the files without the need to erase all files and re-copy them.

  • several parts of your question aren't clear to me. do you want last-used (accessed/read) or last-modified (mtime)? keep in mind that filesystems may be mounted with a noatime flag, preventing atime updates. Also, what tools can we assume, when you say "no additional tools"? – Jeff Schaller Feb 15 '16 at 19:41
1

On the assumption (based on the [linux] tag) that you have bash available, as well as the stat and sort commands; on the further assumption that you want to sync the most-recently-modified files first (see man stat for other timestamp options), then here is a bash script that will loop through all the files in the current directory (for f in * is the key line for that), gathering their last-modified timestamps into an array, then it loops through the sorted timestamps and prints -- a sample! -- rsync command for each file (currently has timestamp debugging information attached as proof).

You'll have to adjust the rsync command for your particular situation, of course. This script will output rsync commands for every file in the current directory; my suggestion would be to either execute these rsync's "blindly", letting the ones at the end fail, or to put them into a script to execute separately.

This script does not attempt to optimize the space utilization of the destination in any way -- the only ordering it does is the last-modification timestamp (and the arbitrary ordering of the associative array in case there are multiple files modified in the same second).

#!/usr/bin/env bash
declare -A times

# gather the files and their last-modified timestamp into an associative array,
# indexed by filename (unique)
for f in *
do
  [ -f "$f" ] && times[$f]=$(stat -c %Y "$f")
done

# get the times in (unique) sorted order
for times in ${times[@]}
do
  echo $times
done | sort -run | while read t
do
  # then loop through the array looking for files with that modification time
  for f in "${!times[@]}"
  do
    if [[ ${times[$f]} = $t ]]
    then
      echo rsync "$f" -- timestamp ${times[$f]}
    fi
  done
done
  • Thanks for the help. Your assumptions were correct. Altough there is still something that can be improved: Your solution is based on the time information inside a single directory. This means if one file inside a directory with 100 files is changed, the entire directory gets a new time and is considered to sync at highest priority (first). Is it possible to decide for every single file (recursively)? In the example above, only the changed file is synced at highest priority and all other files inside the folder remain their time and are sync later (if there is still space). – PhilippS Feb 23 '16 at 12:40
  • Good to hear; good catch on the directory timestamp. I've updated the answer to add a files-only test in the gathering for loop, to avoid that particular issue. I would suggest wrapping this script with cd calls to the various directories you'd like to sync, or calling it with a directory parameter, or perhaps a find .... -type d in order to sync multiple directories. – Jeff Schaller Feb 23 '16 at 13:30
1

You should sort your files by modification date and then pass the names in that order to rsync for copying. You can also use cp instead, but rsync is more efficient.

There is already a thread about how to do this on superuser.com: https://superuser.com/a/208749/554082

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.