In the long description of the many variants of how to include and exclude files, there seems to be one most important sentence:
A given file is excluded by the file selection system exactly when the first matching file selection condition specifies that the file be excluded.
According to this, you need an include option for the files you want first, so they are matched there, and the first match can no longer be an exclude. The, have an exclude option to match everything. The files matched before will not be affected, but everything else is excluded now:
duplicity ... --include "**/*/*.txt" --exclude "**/*" src dest
For your example, that would be
duplicity /source-path --include "**/*/*.txt" --include "**/*/*.xml" --exclude "**/*" /dest-path
To understand which files are matched, you can use dry runs, not changing anything, with logging to list matched files:
duplicity --dry-run --no-encryption -v info --include ... --exclude ... src dest
duplicity accepts the same file selection options rdiff-backup does,
including --exclude, --exclude-filelist-stdin, etc.
When duplicity is run, it searches through the given source directory
and backs up all the files specified by the file selection system. The
file selection system comprises a number of file selection conditions,
which are set using one of the following command line options:
[ ... ]
Each file selection condition either matches or doesn't match a given
file. A given file is excluded by the file selection system exactly
when the first matching file selection condition specifies that the
file be excluded; otherwise the file is included.