SCP : copy the directory but only with specific files

Question

I am trying to copy the directory but only with the specific files.

sudo scp -r user@ip:'ls /nas1/logs/tomcat/dev-main/dev-main*|grep localhost_access_log.2023-09-21.txt' /home/test

If the files are located like this at remote server,

/nas1/logs/tomcat/dev-main/dev-main-1111/localhost_access_log.2023-09-21.txt    
/nas1/logs/tomcat/dev-main/dev-main-1234/localhost_access_log.2023-09-21.txt

then I want to copy the files like this to the local server.

/home/test/dev-main-1111/localhost_access_log.2023-09-21.txt
/home/test/dev-main-1234/localhost_access_log.2023-09-21.txt

Can I use grep with scp command?

Answer 1

Since rsync is not installed per the comment, you can use tar instead.

ssh user@ip 'cd /nas1/logs/tomcat/ && tar cf - dev-main-*/localhost_access_log.2023-09-22.txt' | tar xf -

Explanation:

• ssh user@ip '...' executes the command ... on the remote host
• In this case, the command to be executed is cd /nas1/logs/tomcat/ && tar cv dev-main-*/localhost_access_log.2023-09-22.txt, which first changes the remote directory to /nas1/logs/tomcat/, then executes tar with a glob that matches the files you want to transfer
• The tar command, with the c option with f - for the archive to be dumped on standard output (without the f option, it would send it to the default tape device or stdout depending on the tar implementation and/or environment) which here is a pipe to the local tar command.
• The tar xf - part reads the archive from its standard input (the other end of the pipe) and extracts it, preserving the directory structure and some other metadata (add the p option to preserve even more metadata).

You should execute the above command in /home/test/.

Answer 2

You can't put commands into an scp path, but it looks like you want a set of files that can be matched with a standard glob (wildcard pattern) and there are other tools that will retain the source path.

Here's one:

rsync -avR 'user@ip:/nas1/logs/tomcat/./dev-main/dev-main-*/localhost_access_log.2023-09-21.txt' /home/test/