Recursively "find" file names containing string and symlink files in another directory

Question

I am trying to symlink a set of specific files from a project I'm working on.

There is a known string in each of the filenames I wish to symlink.

Here is what I have so far:

ln -s find ~/path/to/src/ -name "*stringtomatch*" find ~/path/to/dest

I have the directory structure setup in the destination to match the source, but it's just directories, so I don't mind deleting those if it's easier to write a command that for an empty destination.

Update:

I have now accepted a working answer and I want to share some context so that a similar use case might find a solution more easily.

I perform the majority of my coding in Netbeans. When I am building packages for a project I tend to name all of the associated files so that they have part of the file name in common. This allows me to easily find my own package files as I move around within the project. However in my current project this is very time consuming due to the large volume of files and directories involved.

What I have now is a separate project defined for each of my own packages which show only the files for that package while maintaining the master project hierarchy.

By building separate package projects that use symlinks to my package files within the master project, I have effectively created what I believe to be the perfect solution where there doesn't seem to be one available within the Netbeans IDE in its current form.

Each sub-project does nothing but allow me to work on a subset of files relating only to itself which really makes my time at the keyboard more efficient.

I believe Eclipse has this feature in-built, though I do not have Eclipse.

So, albeit a compromise I believe that this workaround for Netbeans is as clean a solution I could achieve today. It's a huge bonus that it works as well, if not better than I had anticipated.

I had expected to run manual synchronisations of the master after edits to the sub-projects. This is not the case, the master still maintains automatic synchronisation.

Answer 1

If you already have the necessary directories created in the target location, then with the GNU implementations of find and xargs, it's not too difficult:

find ~/path/to/src -name "*stringtomatch*" -printf "%P\0" |
  xargs -r0 --replace ln -s ~/path/to/src/'{}' ~/path/to/dest/'{}'

The -printf "%P\0" outputs the path with the source directory parameter removed, and ends with a null byte.

xargs then reads the find output.

-0 indicates the parameters to use are null-separated.

--replace tells xargs to replace {} with the argument (also implies running one command per argument).

Finally the command to replace arguments into is given.

Answer 2

I created a short script to do this, with a nice output which should be easy check the results. It doesn't need to have the destination directory structure created. Use as follows:

$ ./recursive-symlink.sh --help
Usage:
  ./recursive-symlink.sh <source_path> <dest_path> <find_args...>

To show its usage, let's say I have the following files/dirs at the begining:

├── recursive-symlink.sh*
└── src/
    ├── dir1/
    │   ├── file_A_misc.txt
    │   └── file_B_sub.txt
    ├── dir3/
    │   ├── file_A3.txt
    │   ├── file_C.txt
    │   └── subsub_dir/
    │       ├── file_Asubsub.txt
    │       └── file_D.txt
    ├── dir_A/
    │   └── should_be_empty.dat
    ├── file_A.txt
    └── file_B.txt

If I run:

$ find -name '*_A*'
./src/file_A.txt
./src/dir3/file_A3.txt
./src/dir3/subsub_dir/file_Asubsub.txt
./src/dir_A
./src/dir1/file_A_misc.txt

I can see which files would be linked. I then run the script like this:

$ ./recursive-symlink.sh src/ dest/ -name '*_A*'
src/file_A.txt
mkdir: created directory 'dest'
'dest/file_A.txt' -> '../src/file_A.txt'

src/dir3/file_A3.txt
mkdir: created directory 'dest/dir3'
'dest/dir3/file_A3.txt' -> '../../src/dir3/file_A3.txt'

src/dir3/subsub_dir/file_Asubsub.txt
mkdir: created directory 'dest/dir3/subsub_dir'
'dest/dir3/subsub_dir/file_Asubsub.txt' -> '../../../src/dir3/subsub_dir/file_Asubsub.txt'

src/dir_A
'dest/dir_A' -> '../src/dir_A'

src/dir1/file_A_misc.txt
mkdir: created directory 'dest/dir1'
'dest/dir1/file_A_misc.txt' -> '../../src/dir1/file_A_misc.txt'

My final state will be then:

├── recursive-symlink.sh*
├── src/
│   ├── dir1/
│   │   ├── file_A_misc.txt
│   │   └── file_B_sub.txt
│   ├── dir3/
│   │   ├── file_A3.txt
│   │   ├── file_C.txt
│   │   └── subsub_dir/
│   │       ├── file_Asubsub.txt
│   │       └── file_D.txt
│   ├── dir_A/
│   │   └── should_be_empty.dat
│   ├── file_A.txt
│   └── file_B.txt
└── dest/
    ├── dir1/
    │   └── file_A_misc.txt -> ../../src/dir1/file_A_misc.txt
    ├── dir3/
    │   ├── file_A3.txt -> ../../src/dir3/file_A3.txt
    │   └── subsub_dir/
    │       └── file_Asubsub.txt -> ../../../src/dir3/subsub_dir/file_Asubsub.txt
    ├── dir_A -> ../src/dir_A/
    └── file_A.txt -> ../src/file_A.txt

You can see that dest directory is created automatically, as well all recursive subdirectories, and on the dest dir, only the files that matched the *_A* pattern were linked.

---

Here the script source code:

#!/bin/bash

verbose='-v'  # you may comment this line

if [ "$1" == '-h' ] || [ "$1" == '--help' ] || [ $# -lt 3 ]
then
    echo "Usage:"
    echo "  $0 <source_path> <dest_path> <find_args...>"
    exit
fi

src="${1%/}" ; shift
dest="${1%/}" ; shift
relflag='' ; [ "${src:0:1}" != '/' ] && relflag='-r'

find "$src" \( "$@" \) -print0 |
    while IFS= read -r -d '' f
    do
        base_fname="${f#$src}"
        [ "$verbose" ] && echo "${f}"
        dest_ln="$dest/${base_fname#/}"
        dest_dir="$(dirname "$dest_ln")"
        mkdir -p $verbose "$dest_dir"
        ln $relflag -s $verbose -t "$dest_dir" "$f"
        [ "$verbose" ] && echo
    done

Answer 3

With zsh (assuming the target directories already exist):

autoload zmv # best in ~/.zshrc
zmv -Ls ~/path/to/src/'(**/)(*stringtomatch*)' ~/path/to/dst/'$1$2'

If your ln implementation is the GNU one, you can use its -r option to make relative symlinks (here also -v for verbose).

zmv -Ls -o-rv ~/path/to/src/'(**/)(*stringtomatch*)' ~/path/to/dst/'$1$2'