I have two remote servers. One that I am currently connected to and one that I am trying to copy a lot of files to (

I have a series of files I need to copy within various directories of the format:


And this continues for output/1 to output/40 folders. All the folders inside are identical and all the filenames inside will all contain similar strings, just with slight differences depending on the folder they are in.

I want to copy all the files that contain a1 from any directory to a folder in a remote server:


Similarly, I want to do this for all b1, c1, c2 etc. files and copy them to their respective places on the remote server.

I cannot seem to find a way to do this that doesn't involve writing multiple lines of code.

I have tried

cd /opt/DR/output/1/a/
scp -r -v *a1* root@

which works but I want to copy all the *a1* csv files rather than having to do them one by one.

I have looked into globbing but don't think it can be used for my case. I have also looked at using paramiko/glob for python but I couldn't get that to work either. Ideally, I would like to do this with a bash shell script, but a python script would also work.

Hope this makes sense. Any help would be greatly appreciated. I can copy via SFTP or SCP.

  • Do you have any list of those b1, c1, c2, etc?
    – DevilaN
    Feb 4, 2020 at 11:41
  • 1
    "for all b1, c1, c2 etc. files" I don't see the pattern. Is it that directory /1/a/ contains *a1* files, and /99/z/ would hypothetically contain *z99* files? If not, please clarify how we know what your "etc." is to represent.
    – roaima
    Feb 4, 2020 at 11:47

1 Answer 1


It's simple bash script for your task. Just replace BASE string on your directory in local machine, and replace SSH_BASE string on your remote machine. I hope this helps. nesting is 2

 #!/usr/bin/env bash

for i in `ls ${BASE}`
    if [ -d ${BASE}/${i} ]; then
        for j in `ls ${BASE}/${i}`
            if [ -d ${BASE}/${i}/${j} ]; then 
             ssh root@ "mkdir -p ${SSH_BASE}/${i}/${j}"
             scp ${BASE}/${i}/${j}/*.* root@${SSH_BASE}/${i}/${j}/
  • Please do not parse ls. Use at least ls -1 for more robustness. Better: for i in "$BASE"/*; do if the contents is not too much, otherwise while IFS= read -r i; do ...; done < <(ls -1 "$BASE"). Feb 4, 2020 at 14:10
  • When you'll run ssh command inside WHILE IFS, your script may stops. Not good idea. MB this: for I in "$BASE"/*, really better than my code, but no sure
    – spybull
    Feb 4, 2020 at 14:51
  • For the ssh command to work in a while-loop, stdin must be redirected from /dev/null. This could be achieved by explicilty redirecting with < /dev/null, or using the parameter -n to the ssh command, which does the same. Feb 4, 2020 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.