I've around 40k lines of file with paths which I need to take size from remote site (using rsh & du -scL command). I tried with while read line but due to remote connection, it exits after 100+ lines. So I tried to copy all the lines in to a file with du -scL and input the file in to one rsh but again it's crashed saying 'command too long'. I need to do a script which calc the size of all these paths from remote site using rsh and du.

for line in `cat $destbang1`
  rsh vnc.<remotesite> du -sL $line | awk '{print $1}' >> /tmp/size1.txt
total=`gawk '{ sum += $1 }; END { print sum}' /tmp/size1.txt`
echo $total
  • You would be better served using find with the exec flag and so something like find -type d -exec du '{}' \; – Raman Sailopal Sep 11 '17 at 10:17
  • Thanks Raman for the suggestion.But does that can process 40k lines in du via rsh? .I tried while read with -exec and crashed in 100-200 lines.I also tried something like rsh <remotemachine> du -scL cat /file/which/contain/40kpath/lists.txt but it exited with "command too long" – Ratheesh P Sep 11 '17 at 12:36
  • 1
    Is there any reason you're not using ssh. – Raman Sailopal Sep 11 '17 at 12:39
  • 1
    Don't do 100 rsh or ssh commands without any delay between them; each one creates a socket that remains unavailable for a few minutes after it's closed, and if enough of these build up you won't be able to make any more connections for awhile. Instead, get the script working on your local system, show us the code, and we'll help you copy it and execute it on the remote system. – Mark Plotnick Sep 11 '17 at 20:54
  • 1
    If you are asked for code then do not show it in a comment; add it to your question instead. – Hauke Laging Sep 17 '17 at 15:03

You can use xargs for creating several rsh calls with a low enough number or arguments.

Assuming the remot and local limit are the same:

xargs --delimiter='\n' rsh /path/to/script.sh <lists.txt

You can run the entire process with a single rsh remote shell:

tr '\012' '\0' < "$destbang1" |
    rsh vnc.remotesite du -sL --files0-from=- |
    awk '{sum += $1} END {print sum}'

This requires that your du command understands the --files0-from option and that none of the file names in your $deskbang1 file contains an embedded newline.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.