0

I have two directories that look something like the following but with many more files.

folder1/pic1.png
folder1/test/readme.txt

folder2/guest.html
folder2/backup/notes.txt

I want to "merge" these two so all the contents of folder2 end up in folder1 and folder2 gets removed. They are on the same filesystem and disk (ext4). I know all of the files are unique, would mv work fine here?

0

The "rsync" command is useful for this. I do something like this:

rsync -PHACcviuma --copy-unsafe-links --exclude="*~"  folder2/  folder1/  &&  rm -fr folder2

All the flags are documented in the rsync man page; basically rsync won't replace newer files with older ones, and won't bother to copy any files that are duplicated in the destination. Otherwise it will copy things with the original metadata (timestamp, permissions, etc) preserved.

The rsync program will also include "hidden files" (names starting with "."), backups (ending with "~", etc) so I use the --exclude option to skip certain uninteresting file patterns.

  • I don't want to copy, I want to move the files. Unless I'm missing something rsync will make copies of everything and disk intensive. I just want to merge folders of different files into one folder. – RansuDoragon Jan 6 at 14:25
  • Well, that's what the "rm -fr folder2" at the end of the line is for. – s'kiddie cat Jan 7 at 15:05
  • I thought I read that moving files was faster if done on the same filesystem. From what I understand 'rsync' copies each file but 'mv' actually just edits where the file is in the filesystem instead of rewriting the whole file IF it's the same filesystem. I just wanted to double check that I wasn't misunderstanding what happens when using 'mv' to move files inside one folder to another folder. – RansuDoragon Jan 8 at 9:08
  • Yes, mv'ing is faster if on the same fs, since it's just an update of the directory entry. If you're sure any duplicated files are really duplicates (not updated files with the same name) then "mv -f -r ..." would work for everything except the hidden files, then "mv -f .*" would get the hidden files, then you could handle any hidden directories individually. Or if you're going to do this often, then a script to handle all the "edge cases" would be fast and effective. Not worth it if this is a one-time effort, though. – s'kiddie cat Jan 9 at 9:29
  • I've used a program called 'mergerfs' from github.com/trapexit/mergerfs This basically 'combines' in software a bunch of other storage locations (like a JBOD) to one single mount point. This means that because of how I setup 'mergerfs' to store files across my connected hard drives that I have a bunch of folders that are named the same. This is causing issues with 'mv' wanting to overwrite folders and then failing because "directory not empty". Instead of writing a script to handle this with mv, I've switched to using rsync. – RansuDoragon Jan 9 at 13:30
1

Yes, mv works here.

$ mv -i folder2/* folder1/

Note the -i flag is to add some safety.

  • The glob also grabs any hidden folders or files marked with a dot? – RansuDoragon Jan 6 at 9:17
  • Nope, the * doesn't match those hidden files/dirs whose name start with a dot. If you want to mv those hidden files/dirs, replace * with {,.[^.],..?}*. – Charles Jan 6 at 9:25
  • I've used a program called 'mergerfs' from github.com/trapexit/mergerfs This basically 'combines' in software a bunch of other storage locations (like a JBOD) to one single mount point. This means that because of how I setup 'mergerfs' to store files across my connected hard drives that I have a bunch of folders that are named the same. This is causing issues with 'mv' wanting to overwrite folders and then failing because "directory not empty". I'm switching the answer because rsync works better in this case. It takes longer because it's copying but at least it's more reliable. – RansuDoragon Jan 9 at 13:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.