34

I have created the following script that move old days files as defined from source directory to destination directory. It is working perfectly.

#!/bin/bash

echo "Enter Your Source Directory"
read soure

echo "Enter Your Destination Directory"
read destination 

echo "Enter Days"
read days



 find "$soure" -type f -mtime "-$days" -exec mv {} "$destination" \;

  echo "Files which were $days Days old moved from $soure to $destination"

This script moves files great, It also move files of source subdirectory, but it doesn't create subdirectory into destination directory. I want to implement this additional feature in it.

with example

/home/ketan     : source directory

/home/ketan/hex : source subdirectory

/home/maxi      : destination directory

When I run this script , it also move hex's files in maxi directory, but I need that same hex should be created into maxi directory and move its files in same hex there.

12 Answers 12

33

I know find was specified, but this sounds like a job for rsync.

Examples

Mirror files with same directory structure (source remains in tact):

rsync -axuv --progress Source/ Target/

Move files with same directory structure (removing from source and removing empty directories):

rsync -axuv --prune-empty-dirs --remove-source-files --progress Source/ Target/

Move files of a particular file-type (example):

rsync -rv --include '*/' --include '*.js' --exclude '*' --prune-empty-dirs Source/ Target/

Move files resulting from an advanced find search:

cd "$source" &&
  rsync -av --remove-source-files --prune-empty-dirs --progress --files-from <(find . -type f -mtime -$days) . "$destination"

A note about file removal

There are options for rsync that can ensure your destination directory mirrors your source directory, which then remove files in the destination that are no longer in your source (i.e. --delete-before, --delete-after, --delete-during, and --delete-delay.

You can also remove files from allow removing files from the source directory when they have been moved from the destination directory, i.e. --remove-source-files.

This is use case dependent, so how you handle that is up to you.

4
  • much cleaner than the others solutions :)
    – Guillaume
    May 31, 2017 at 12:08
  • 4
    I found that --remove-source-files was helpful, which effectively causes files to be moved instead of copied. This is an awesome use of rsync.
    – Tom
    Aug 30, 2017 at 5:27
  • The only problem I see in this approach is that I did not find a way to specify a date or a time window for rsync to consider (e.g., move files older than 3 months). Jun 15 at 14:34
  • @LuisTalora You could try using --include-from combined with a find search piped to a grep that filters by timestamp. Jun 15 at 16:56
23

Instead of running mv /home/ketan/hex/foo /home/maxi, you'll need to vary the target directory based on the path produced by find. This is easier if you change to the source directory first and run find .. Now you can merely prepend the destination directory to each item produced by find. You'll need to run a shell in the find … -exec command to perform the concatenation, and to create the target directory if necessary.

destination=$(cd -- "$destination" && pwd) # make it an absolute path
cd -- "$source" &&
find . -type f -mtime "-$days" -exec sh -c '
  mkdir -p "$0/${1%/*}"
  mv "$1" "$0/$1"
' "$destination" {} \;

Note that to avoid quoting issues if $destination contains special characters, you can't just substitute it inside the shell script. You can export it to the environment so that it reaches the inner shell, or you can pass it as an argument (that's what I did). You might save a bit of execution time by grouping sh calls:

destination=$(cd -- "$destination" && pwd) # make it an absolute path
cd -- "$source" &&
find . -type f -mtime "-$days" -exec sh -c '
  for x do
    mkdir -p "$0/${x%/*}"
    mv "$x" "$0/$x"
  done
' "$destination" {} +

Alternatively, in zsh, you can use the zmv function, and the . and m glob qualifiers to only match regular files in the right date range. You'll need to pass an alternate mv function that first creates the target directory if necessary.

autoload -U zmv
mkdir_mv () {
  mkdir -p -- $3:h
  mv -- $2 $3
}
zmv -Qw -p mkdir_mv $source/'**/*(.m-'$days')' '$destination/$1$2'
8
  • for x do, you've got a missing ; there :). Also, I have no idea what you wanted to achieve with $0 but I'm quite convinced it would be sh :). Oct 12, 2014 at 19:02
  • @MichałGórny for x; do is technically not POSIX-compliant (check the grammar), but modern shells allow both for x do and for x; do; some old Bourne shells didn't grok for x; do. On modern shells, with sh -c '…' arg0 arg1 arg2 arg3, arg0 becomes $0, arg1 becomes $1, etc. If you want $0 to be sh, you need to write sh -c '…' sh arg1 arg2 arg3. Again, some Bourne shells behaved differently, but POSIX specifies this. Oct 12, 2014 at 19:11
  • pushd seems like a better choice that cd, as it less intrusive into the current environment.
    – jpmc26
    Aug 15, 2018 at 19:52
  • This command creates directories for complete path, that is it creates a directory for the matching file as well. I wish to copy directory structure only upto the matching file. Jun 26, 2020 at 7:55
  • @Gilles'SO-stopbeingevil', thanks, this is very useful btw, i used sed -i.bak for many files, i want to move those .bak files to another backup folder, by using your script, all .bak files moved to my back folder successfully how can i change the script to remove .bak in file name, so in backup folder, filename is same as original one?
    – jerry
    Mar 13, 2021 at 12:56
4

you could do it using two instances of find(1)

There's always cpio(1)

(cd "$soure" && find … | cpio -pdVmu "$destination")

Check the arguments for cpio. The ones I gave

3
  • 1
    This will break on any filenames with whitespace.
    – Chris Down
    Dec 21, 2012 at 17:26
  • 1
    This copies files instead of moving them. Dec 22, 2012 at 0:01
  • May not be a perfect answer, but it helped me to move files preserving paths in a more or less straight forward manner (I have enough space to copy the files then delete). Upvoted
    – AhHatem
    Apr 27, 2014 at 10:01
4

It's not as efficient, but the code is easier to read and understand, in my opinion, if you just copy the files and then delete afterwards.

find /original/file/path/* -mtime +7 -exec cp {} /new/file/path/ \;
find /original/file/path/* -mtime +7 -exec rm -rf {} \;

Notice: Flaw discovered by @MV for automated operations:

Using two separate operations is risky. If some files become older than 7 days while the copy operation is done, they won't be copied but they will be deleted by the delete operation. For something being done manually once this may not be an issue, but for automated scripts this may lead to data loss

2
  • 3
    Using two separate operations is risky. If some files become older than 7 days while the copy operation is done, they won't be copied but they will be deleted by the delete operation. For something being done manually once this may not be an issue, but for automated scripts this may lead to data loss.
    – MV.
    Dec 28, 2016 at 5:47
  • 3
    The easy solution to this flaw is to run find once, save the list as a text file, and then use xargs twice to do the copy and then the delete. Feb 21, 2018 at 19:48
3

You can do this by appending the absolute path of the file returned by find to your destination path:

find "$soure" -type f -mtime "-$days" -print0 | xargs -0 -I {} sh -c '
    file="{}"
    destination="'"$destination"'"
    mkdir -p "$destination/${file%/*}"
    mv "$file" "$destination/$file"'
3
  • Chris now its moves whole home into maxi
    – KK Patel
    Dec 21, 2012 at 15:11
  • @K.KPatel No, it doesn't. It merely keeps your directory structure.
    – Chris Down
    Dec 21, 2012 at 17:26
  • 1
    This breaks if $destination contains special characters, because it undergoes expansion in the inner shell. Maybe you meant destination='\'"$destination"\''? That still breaks on '. Also, this creates files such as /home/maxi/home/ketan/hex/foo instead of /home/maxi/hex/foo. Dec 22, 2012 at 0:03
2

Better (fastest & without consuming storage space by doing copy instead of move), also is not affected by the file-names if they contain special characters in their names:

export destination
find "$soure" -type f "-$days" -print0 | xargs -0 -n 10 bash -c '
for file in "$@"; do
  echo -n "Moving $file to $destination/"`dirname "$file"`" ... "
  mkdir -p "$destination"/`dirname "$file"`
  \mv -f "$file" "$destination"/`dirname "$file"`/ || echo " fail !" && echo "done."
done'

Or faster, moving a bunch of files in the same time for multi-CPU, using the "parallel" command:

echo "Move oldest $days files from $soure to $destination in parallel (each 10 files by "`parallel --number-of-cores`" jobs):"
function move_files {
  for file in "$@"; do
    echo -n "Moving $file to $destination/"`dirname "$file"`" ... "
    mkdir -p "$destination"/`dirname "$file"`
    \mv -f "$file" "$destination"/`dirname "$file"`/ || echo " fail !" && echo "done."
  done
}
export -f move_files
export destination
find "$soure" -type f "-$days" -print0 | parallel -0 -n 10 move_files

P.S.: You have a typo, "soure" should be "source". I kept the variable name.

2

This is less elegant but easy if the number / size of files isn't too great

Zip your files together into a zip archive, and then unzip at the destination without the -j option. By default, zip will create the relative directory structure.

0

I am using it this way

cp -r source/ destination/
find destination/ -not -path "*/mypattern/*.py" -delete

Basically copy everything from source to destination and delete everythings other than the required stuff.

-1

Try this way:

IFS=$'\n'
for f in `find "$soure" -type f -mtime "-$days"`;
do
  mkdir -p "$destination"/`dirname $f`;
  mv $f "$destination"/`dirname $f`;
done
-1

Because there seem to be no really easy solution to this and I need it very often, I created this open source utility for linux (requires python): https://github.com/benapetr/smv

There are multiple ways how you could use it to achieve what you need but probably most simple would be something like this:

 # -vf = verbose + force (doesn't stop on errors)
smv -vf `find some_folder -type f -mtime "-$days"` target_folder

You can additionally run it in dry mode so that it doesn't do anything but print what it would do

smv -rvf `find some_folder -type f -mtime "-$days"` target_folder

Or in case that list of files is too long to fit into argument line and you don't mind executing python for every single file, then

find "$soure" -type f -mtime "-$days" -exec smv {} "$destination" \;
-1

Here's what I've been using, with find, tar, and rm. Replace find arguments with arguments you would need, but retain the -type f, files only option.

cd srcdir        # OR pushd srcdir

find . -mtime +7 -type f |  while read fn; do echo Moving $fn; tar cf - "$fn" | ( cd destdir; tar xf - );  rm -f "$fn"; done

# popd 

(Skip the "echo Moving $fn" command to execute the job silently.) The above method may leave empty directories in the source tree. One could use

find srcdir -empty -type d -delete 

to remove empty directories.

-3
#!/bin/bash

# '+' here shows that 'find' looking for files 45 and more days older from NOW
# can be replaced with '-' (-45), to search for files with modification date from NOW to 45 days(max)
days='+45'

source=/var/log

destination=/root/logsbackups

# searching can take some time, so inform user/script what is happening
echo "\nSearching for files..."

# collecting files list as Array to variable
LIST_OF_FILES=(`find $source -type f -mtime $days`)

echo "Moving files..."

# process collected file paths Array for proper moving
for file in ${LIST_OF_FILES[@]}; do

  # real full path to file (without filename)
  filepath=$(dirname $file)
  
  # full file name (without path)
  filename=$(basename $file)

  # log info (unnecessary)
  echo "Moving to $destination$filepath$filename"
  
  # make sure that target directory exists
  mkdir -p $destination$filepath
  
  # moving file
  mv -f $file $destination$filepath$filename

done
4
  • Consider testing this on a collection of files with names containing spaces and filename globbing characters. Since you don't quote any of your expansions, the shell would perform splitting and filename globbing on the output of find, and then again later when you fail to quote $file, $filepath and $filename.
    – Kusalananda
    Jun 26 at 19:18
  • @Kusalananda good point. But still works for me. Tested on Debian, and right now works for transfer mail files (300K) on CentOS virtual server with limited (10GB) storage space. You still can suggest more elegant and idiot proof code.
    – Alexey
    Jun 26 at 19:43
  • 1
    (1) With all due respect, nobody cares if the script is working for you using your filenames, which are all-alphanumeric (foo, bar, ds9, etc.) There’s no way LIST_OF_FILES=(`find $source  -type f  …`) works correctly for all filenames (or even a tricky value for $source).  (2) We don’t like code-only answers.  Please explain what you are doing (and how it is different from previous answers).  (3) See Why is looping over find's output bad practice? Jun 26 at 20:02
  • @g-man-says-reinstate-monica with all due respect, speak for yourself. Where those 'we' are? I do not care about, your personal opinion, for my way to share real, working, and finished code, what can be used with minor changes, not like most sketches, in internet in general, or on this page particular. I am not forcing any one to use it, but I am guessing - if i had to make such script, and google some of it parts, it can be useful for someone, who not picky as 'WE-GROUP-PEOPLE'. About 'loop-in-find-result': for every task - use proper tools. Don't overcomplicate, please.
    – Alexey
    Jun 26 at 20:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.