Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

We have multiple deployment of an application on servers such as app00, app01 and so on. I need to copy a single log file from all these servers onto my local mac so I can perform some grepping and cutting.

I used csshX for viewing this file but I cannot find an equivalent for scp. I basically want two things:

  1. Ability to connect to n numbers of such servers and copy the file
  2. Avoid naming conflicts locally perhaps by prefixing the log file with the server hostname

How do I do this?

share|improve this question
1  
I would probably write a script for it myself. Is that not an option? –  Bernhard May 20 at 11:32
    
I am doing that right now Bernhard. :) I was just hoping there was something like csshX for this also. –  Gaurav May 20 at 11:37

4 Answers 4

This is trivial to do with a little script. For example:

for server in app0 app1 app4 app5 appN; do
    scp user@$server:/path/to/log/file /local/path/to/"$server"_file
done

The above will copy the file from each of the servers sequentially and name it SERVERNAME_file. So, the file from app0 will be app0_file etc. You can obviously change the names to whatever you would like.

share|improve this answer
2  
Put & at the end of the scp command and a wait at the end, and you have concurrency at no extra cost. –  l0b0 May 20 at 17:21
2  
@l0b0 I don't know if that's desired. I like this approach because I can know if and which files failed to copy. Good tip though, thanks. –  terdon May 20 at 17:25

Use GNU parallel:

parallel -j0 scp {}:/remote_path file_from_{} ::: host1 host2 host3 # and so on

Unlike solutions that use a for, this will run all the downloads in parallel

share|improve this answer
remote_path="/path/to/file"
local_target_dir="/path/to/dir"
hosts=(app00 app01)
for host in "${hosts[@]}"; do
    scp "$host":"$remote_path" "$local_target_dir"/filename."$host"
done
share|improve this answer
    
Thanks a lot, I am new to bash programming and wrote something like that, just a lot dirtier. :) –  Gaurav May 20 at 11:37

If you are able to use python, there is an interesting module that looks to simplify machine administration-ish tasks called fabric: http://docs.fabfile.org/en/latest/tutorial.html

I have wanted to use it but have not got around too it

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.