-1

I have 40 WGS files each one about to 0.1 TB stored in a hard drive next to me. I need to transfer them to my scratch in our computing cluster. I am using windows OS so I am using MobaxTerm for connecting to computing cluster

People say by

rsync -axv --numeric-ids --progress -e "ssh -T -o Compression=no -x" *.file user@path_to_hpc(...):/scratch/your_username/folder...

But I am not able to figure out how to use that when I am in windows

  • How far away is the computing cluster? Would it be an option to bring the HDD with the files to the cluster (and connect it via eSATA or USB 3? Would it be an option to boot your computer into an easy to use live linux drive, for example a USB drive with Ubuntu, where rsync is part of the system? – sudodus Mar 20 at 12:33
  • I found a link in using rsync in windows but very complicated to understand superuser.com/questions/300263/… – Feresh Teh Mar 20 at 12:55
  • I think more than one of the solutions in that link are likely to work. If you find them too complicated, please consider the two alternatives, that I suggested in my first comment. By the way, have you ever used linux? Have you used sftp or Filezilla or WinSCP? -- How fast and reliable is the network connection between your computer and the server? – sudodus Mar 20 at 13:01
  • I used sftp via mobaxterm in my windows that was tooooo slow considering wireless connection here. people says I can access the locally mounted external drive under /mnt in MobaXTerm. I have used linux for a bit so I know something but not too much. I am not finding mnt in mobaxterm – Feresh Teh Mar 20 at 13:11
  • rsync will not be faster than sftp. I would recommend a wired connection for speed and stability, when you intend to transfer 4 TB. -- How far away is the server? – sudodus Mar 20 at 13:15
1

in rsync command line

  • *.file stand for source file (e.g. your local disk)
  • user@path_to_hpc(...):/scratch/your_username/folder... stand for destination.
  • user username
  • path_to_hpc(...) IP or hostname
  • /scratch/your_username/folder... destination paths (e.g. /scratch/fi1d18 )

local files

the best way to proceeed is to

cd /where/r/myfile
  • where /where/r/myfile is local path to you disk.

then

rsync ..options.. *.bam user@remote:/scratch/fi1d18
  • Sorry, I am not able to figure out how to point to my hard derive in this code, how this code knows where are my files? My .bam files are in an external big hard derive next to me. This makes me confused :( – Feresh Teh Mar 20 at 12:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.