rsync only certain files types from specified sub-directories

Question

Here is my source dir structure:

../openwrt
../openwrt/BackupOfPackages
../openwrt/BackupOfSettings
../openwrt/BackupOfImages

I want to use rsync to mirror that to the destination dir. I only want to rsync files in the three sub-directories and I only want to sync files with extension:
.installed from the BackupOfPackages folder
.img.gz from the BackupOfImages folder
.tar.gz from the BackupOfSettings folder

In my source dir I have a mixture of all sorts of files in the basedir "openwrt" and in each of the sub dirs. However as above, I only want to sync specific file types from specific sub-dirs.

Here is my rsync command:

openWrtPath="/mnt/usb/openwrt/"  
ncpPath="/media/myCloudDrive/openwrt"

rsync -vvrit --include='/' --include='BackupOfSettings/' --include='BackupOfSettings/**.tar.gz' --include='BackupOfPackages/' --include='BackupOfPackages/**.installed' --include='BackupOfImages/' --include='BackupOfImages/**.img.gz' --exclude='/*' --delete --delete-excluded $openWrtPath ncp:$ncpPath

When I run the command this is what it shows and hides. It correctly hides opkg.installed and "New Folder" from the basedir but does not hide file types such as .installed from BackupOfSettings and BackupOfImages etc..

Rsync syncs almost everything from the source to the destination?

How do I change my rsync command to only copy files with extension:
.installed to the BackupOfPackages folder
.img.gz to the BackupOfImages folder
.tar.gz to the BackupOfSettings folder
i.e: To correctly hide any file types that should not be transferred to the destination from the various sub-dirs even if they exist in those sub dirs on the source machine.

UPDATE

Right after I posted I think I found the answer.. when I change --exclude='/*' to --exclude='*' it seems to work as I want. Can someone explain the difference between those two excludes?

Does '/*' mean: "Exclude everything in the basedir (openwrt) and this does not apply to sub-directories of the basedir"?

Thank you!

Answer 1

Does '/*' mean: "Exclude everything in the basedir (openwrt) and this does not apply to sub-directories of the basedir"?

Yes, / is the root of the transfer directory and /* matches everything there (files and directories).

You can shorten the includes in your command to:

rsync -ai \
  --include='BackupOf***/' \
  --include='BackupOfImages/**.img.gz' \
  --include='BackupOfSettings/**.tar.gz' \
  --include='BackupOfPackages/**.installed' \
  --exclude='*' \
  --delete --delete-excluded \
  "$openWrtPath" ncp:"$ncpPath"

Here --include='BackupOf***/' means to include all directories starting with BackupOf and its subdirectories (e.g. BackupOfImages/sub1/sub2/). If you don't need to include those subdirectories, change it to BackupOf*/ and you can also change the two **'s of the file extensions to one *.

I used the archive mode -a which includes -t and other options and quoted both variables.

Answer 2

rsync doesn't include a / at the start of the paths it considers. Why? designer's choice.

But there is another issue. You've used ** in your includes to include files in subdirectories. However, the subdirectories themselves are excluded. Try changing the includes and excludes to:

--include='*/' --include='BackupOfSettings/**.tar.gz' --include='BackupOfPackages/**.installed' --include='BackupOfImages/**.img.gz' --exclude='*' --prune-empty-dirs

This is: include all directories, only the files you want, no other files, and drop the directories that are empty.

The other option would be to write:

--include='BackupOfSettings/' --include='BackupOfSettings/**/' --include='BackupOfSettings/**.tar.gz' --include='BackupOfPackages/' --include='BackupOfPackages/**/' --include='BackupOfPackages/**.installed' --include='BackupOfImages/' --include='BackupOfImages/**/' --include='BackupOfImages/**.img.gz' --exclude='*'

In this case, this is including the specific directories, subdirectories thereof, and the wanted files. This variant would be needed if you have a large number of other directories in the source tree that you wish to avoid searching.

Answer 3

To expand on the accepted answer by @Freddy

My source and destination directories:

openWrtPath="/mnt/usb/openwrt/"  
ncpPath="/media/myCloudDrive/openwrt"

My goal is to only sync files with extension .tar.gz from a subfolder in the source directory called: BackupOfSettings. Subfolders in the BackupOfSettings subfolder are also ignored.

Here is a simplified version of the command in the question. Credit here and here for explanation of how rsync will sync a specific file type from a specific sub-dir.

rsync -vvritn --include='BackupOfSettings/' --include='BackupOfSettings/*.tar.gz' --exclude='*' $openWrtPath ncp:$ncpPath

1. To achieve this with rsync you need to first include the sub-directory that contains the files you want: --include='BackupOfSettings/'
2. Then in a separate include you need to define a pattern in that same sub-directory that will match your desired file type: --include='BackupOfSettings/*.tar.gz'
3. Then you need to exclude everything else using: --exclude='*' which matches and excludes files and folders that are not explicitly included.

--exclude='/*' vs --exclude='*'

rsync -vvritn --include='BackupOfSettings/' --include='BackupOfSettings/*.tar.gz' --exclude='/*' $openWrtPath ncp:$ncpPath

If we use --exclude='/*' then the above command will allow all files in the BackupOfSettings sub-folder because this exclude only matches on files and sub-folders in the source directory... it does not match files in sub-folders of the source directory.