142

If I use mv to move a folder called "folder" to a directory that already contains "folder" will they merge or will it be replaced?

11 Answers 11

110

mv cannot merge or overwrite directories, it will fail with the message "mv: cannot move 'a' to 'b': Directory not empty", even when you're using the --force option.


You can work around this using other tools (like rsync, find, or even cp), but you need to carefully consider the implications:

  • rsync can merge the contents of one directory into another (ideally with the --remove-source-files1 option to safely delete only those source files that were transferred successfully, and with the usual permission/ownership/time preservation option -a if you wish)
    but this is a full copy operation, and can therefore be very disk-intensive.
  • Currently preferred option: You can combine rsync's --link-dest=DIR option (to create hardlinks instead of copying file contents, where possible) and --remove-source-files to get a semantic very similar to a regular mv.
    For this, --link-dest needs to be given an absolute path to the source directory (or a relative path from the destination to the source).
    but this is using --link-dest in an unintended way (which may or may not cause complications), requires knowing (or determining) the absolute path to the source (as an argument to --link-dest), and again leaves an empty directory structure to be cleaned up as per 1.
  • You can use find to sequentially recreate the source directory structure at the target, then individually move the actual files
    but this has to recurse through the source multiple times and can encounter race conditions (new directories being created at the source during the multi-step process)
  • cp can create hard links (simply put, additional pointers to the same existing file), which creates a result very similar to a merging mv (and is very IO-efficient since only pointers are created and no actual data has to be copied)
    but this again suffers from a possible race condition (new files at the source being deleted even though they weren't copied in the previous step)

Which of these workarounds (if any) is appropriate will very much depend on your specific use case.
As always, think before you execute any of these commands, and have backups.


1: Note that rsync --remove-source-files won't delete any directories, so you will have to do something like find -depth -type d -empty -delete afterwards to get rid of the empty source directory tree.

  • It sounds like you've tried just one implementation of mv. This answer would be better with a broader truth. Linux, BSD and "real" Unix, or a reference from POSIX or SUS. – Warren Young May 3 '14 at 19:29
  • @WarrenYoung You're right, I only tried the mv implementation used by Debian - the emphasis being on tried, since the manpage doesn't mention this behavior... – n.st May 3 '14 at 23:47
  • 37
    The disadvantage of rsync is that it actually copies the data, rather than just changing the hard link, which is potentially resource intensive if you're dealing with a lot of data. – Jonathan Mayer Sep 13 '14 at 18:53
  • rsync can delete stuff with --delete. – Keith Nov 9 '16 at 18:16
  • 6
    @Keith Note that --delete only deletes files in the destination directory that don't exist in the source directory. – n.st Nov 9 '16 at 18:18
85
rsync -av /source/ /destination/
(after checking)
rm -rf /source/
  • Will this remove the source files like in the comment by n.st? – Dominique May 3 '14 at 19:01
  • 3
    No, I would prefer to make it in two steps for safety reasons. Merged and removed source is unreversible. Additon step in n.st anwer is also needed (to remove directories). – fazie May 3 '14 at 19:10
  • 3
    --remove-source-files has the advantage of only removing files that were transferred successfully, so you can use find to remove empty directories and will be left with everything that wasn't transferred without having to check rsyncs output. – n.st May 3 '14 at 23:51
  • 4
    But it is not moving actually - the speed impact is huge, if big files are involved. – Alex Mar 15 '16 at 15:12
  • But you cannot perform pure move and merging actually. – fazie Mar 15 '16 at 19:58
62

You can use the -l option of the cp command, which creates hard links of files on the same filesystem instead of full-data copies. The following command copies the folder source/folder to a parent folder (destination) which already contains a directory with the name folder.

cp -rl source/folder destination
rm -r source/folder

You may also want to use the -P (--no-dereference - do not de-reference symbolic links) or -a (--archive - preserve all metadata, also includes -P option), depending on your needs.

  • 7
    @rautamiekka: I assume you are asking the reason for using hard links. If you don't know what hard links are and why you should use them, then you probably shouldn't take this route. However, creating hard links does not do a full copy, so this operation would take orders of magnitude less time than a full copy. And, you would use hard links rather than soft links so that you can delete the source files and still have the correct data instead of pointers to invalid paths. And cp rather than rsync since every system has cp and everyone has familiarity with it. – palswim Feb 16 '16 at 18:24
  • 6
    This solution's brilliance may be looked over for not being the accepted answer. It is an elegant solution. You get the merge ability of cp with the operation time of mv. – TheHerk Dec 9 '16 at 14:41
  • 2
    if you know that you don't need to move files that already exist on the destination you also want to add -n – ndemou May 8 '17 at 22:32
  • 1
    @Ruslan: This is true, but you can't move without a copy across filesystems with any method. Even mv /fs1/file /fs2/ (across filesystems) will perform a copy and then a delete. – palswim Jun 29 '17 at 19:37
  • 1
    Right, but while mv will work (provided the target dir doesn't yet exist) even if not "efficiently" or whatever you call it, cp -rl will fail. – Ruslan Jun 29 '17 at 19:47
21

I'd recommend these four steps:

cd ${SOURCE}; 
find . -type d -exec mkdir -p ${DEST}/\{} \; 
find . -type f -exec mv \{} ${DEST}/\{} \; 
find . -type d -empty -delete

or better yet, here's a script that implements semantics similar to mv:

#!/bin/bash

DEST="${@:${#@}}"
ABS_DEST="$(cd "$(dirname "$DEST")"; pwd)/$(basename "$DEST")"

for SRC in ${@:1:$((${#@} -1))}; do   (
    cd "$SRC";
    find . -type d -exec mkdir -p "${ABS_DEST}"/\{} \;
    find . -type f -exec mv \{} "${ABS_DEST}"/\{} \;
    find . -type d -empty -delete
) done
  • Args are SOURCE, DEST – schuess Jan 4 '16 at 14:32
  • This looks pretty useful. I'm tempted to use it for cleaning up my hard drive. Can any other experts comment on it before I entrust a bunch of backups to this script? :-) – LarsH Jun 29 '16 at 3:16
  • BTW, if you were wanting to do the equivalent of rsync -u (only update if newer), mv (in some versions at least) can also take the -u option. However in that case you might want to delete non-empty source directories as well as empty ones, to cover the cases where files in the source tree are not newer. @schuess: It looks like there can be multiple SOURCE arguments, if you need that. – LarsH Jun 29 '16 at 3:26
  • 1
    This doesn't handle whitespace well. I gave it a try with some directories with spaces in them and ended up with an infinite series of nested directories. – rofer Sep 30 '17 at 16:01
16

Here is a way that will merge the directories. It is much faster than rsync since it just renames the files instead of copying them and then deleting them.

cd source; find -type f -print0 | xargs -0 -n 1 -I {} mv '{}' 'dest/{}'
  • That's interesting but only vaguely relevant to the topic and not even remotely what the user asked about. – Shadur Sep 5 '14 at 5:14
  • 17
    Actually, Jewel's code does precisely what the user asked for, with the exception of creating missing directories. Perhaps you should look again? – Jonathan Mayer Sep 13 '14 at 18:44
  • 3
    I would add to use "-print0" in find and "-0" in xargs because there are files that have spaces in the names. Also, there is a small problem, if a name contains parenthesis they are not going to be moved. – markuz Aug 18 '15 at 16:33
  • 2
    This is much faster than rsync for a small number of files, but it forks a new process for every file, thus the performance is abysmal with a large number of small files. @palswim's answer does not suffer from this problem. – b0fh Mar 1 '16 at 14:43
  • 2
    The command will fail, if in dest is already a directory with the same name as in source. And the files will be moved to a dest, which is in source. The command does nothing more than mv source/* source/dest/. – ceving Aug 25 '16 at 13:38
4

One way to accomplish this would be to use:

mv folder/* directory/folder/
rmdir folder

As long as no two files have the same name in folder and directory/folder, you will achieve the same result i.e. merging.

  • 3
    How exactly does rm folder work? – JakeGould May 4 '14 at 4:28
  • 5
    @JakeGould Not at all. :) – n.st May 4 '14 at 6:58
  • rm folder -fR always works for me – Octopus Dec 10 '15 at 2:43
  • 2
    Be aware that this will not work for hidden files – b0fh Mar 1 '16 at 14:20
2

For the purest copies, I use the tar (-)B blockread copy method.

example, from source path:

tar cBf - <sourcefolder> | (cd /your/target/folder ; tar xBf -)

this creates an exact copy of the source tree, WITH the owner and permissions intact. And if the target folder exists, the data will be merged. Only files that are already existing will be overwritten.

When the copy action is successful, you can remove the source (rm -rf <source>). Of course this is not an exact move: the data will be copied, until you remove the source.

As option you can be verbose (display on screen the file being copied), with -v: tar cBvf -

  • c: create
  • B: read full block (for pipe read)
  • v: verbose
  • f: file to write
  • x: extract
  • -: stdout/stdin

sourcefolder can also be * (for anything in current folder)

  • Specifying f - to tar is usually unnecessary - the default is to read from stdin/write to stdout. – muru Jun 27 '17 at 8:11
1

Here is a script that worked for me. I prefer mv over rsync, so I use Jewel and Jonathan Mayer's solutions.

#!/bin/bash

# usage source1 .. sourceN dest

length=$(($#-1))
sources=${@:1:$length}
DEST=$(readlink -f ${!#})
for SRC in $sources; do
    pushd $SRC;
    find . -type d -exec mkdir -p ${DEST}/{} \;
    find . -type f -exec mv {} ${DEST}/{} \;
    find . -type d -empty -delete
    popd
done
  • This solution does not properly escape path names, be careful. – user12439 Jun 7 '15 at 5:15
  • @user12439, I'll update the solution if you show me which part to fix. – xer0x Jun 8 '15 at 22:53
1

It is not a good idea to use commands like cp or rsync. For large files, it will take a long time. mv is much faster since it only update the inodes without copying the files physically. A better option is to use the file manager of your operating system. For Opensuse, there is a file manager called Konquerer. It can move files without actually copying them. It has "cut and paste" function like in Windows. Just select all the sub-directories in directory A. Right click and "move into" directory B which may contain sub-directories with the same names. It will merge them. There are also options whether you want to overwrite or rename files with the same name.

  • 1
    OP asks what happens when mv is used. – don_crissti Feb 26 '16 at 13:49
0

Python solution

Since I could not find a satisfactory pre-existing solution, I decided to make a quick Python script to achieve it.

In particular, this method is efficient because it only walks the source file tree once bottom up.

It will also allow you to quickly tweak things like file overwrite handling to your liking.

Usage:

move-merge-dirs src/ dest/

will move all contents of src/* into dest/ and src/ will disappear.

move-merge-dirs

#!/usr/bin/env python3

import argparse
import os

def move_merge_dirs(source_root, dest_root):
    for path, dirs, files in os.walk(source_root, topdown=False):
        dest_dir = os.path.join(
            dest_root,
            os.path.relpath(path, source_root)
        )
        if not os.path.exists(dest_dir):
            os.makedirs(dest_dir)
        for filename in files:
            os.rename(
                os.path.join(path, filename),
                os.path.join(dest_dir, filename)
            )
        for dirname in dirs:
            os.rmdir(os.path.join(path, dirname))
    os.rmdir(source_root)

if __name__ == '__main__':
    parser = argparse.ArgumentParser(
        description='Move merge src/* into dest. Overwrite existing files.'
    )
    parser.add_argument('src_dir')
    parser.add_argument('dest_dir')
    args = parser.parse_args()
    move_merge_dirs(args.src_dir, args.dest_dir)

GitHub upstream.

See also: https://stackoverflow.com/questions/22588225/how-do-you-merge-two-directories-or-move-with-replace-from-the-windows-command

0

This is the command for moving files & folders to other destination:

$ mv /source/path/folder /target/destination/

Remember: mv command will not work if the folder is b̲e̲i̲n̲g m̲e̲r̲ge̲d̲ (i.e. another folder with the same name already exist in the destination) and the d̲e̲s̲t̲i̲n̲a̲t̲i̲o̲n̲ o̲n̲e̲ i̲s̲ n̲o̲t̲ e̲m̲pt̲y.

mv: cannot move '/source/path/folder' to '/target/destination/folder': Directory not empty

If the destination folder is empty, the above command will work fine.

So, in order to merge both folders in any case,
Either do it in 2 commands:

$ cp -rf /source/path/folder /target/destination/
$ rm -rf /source/path/folder

Or combine both as a one-time command:

$ cp -rf /source/path/folder /target/destination/ && rm -rf /source/path/folder

mv = move
cp = copy
rm = remove

-r for directory (folder)
-f force execution

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.