I have created some shell script to put the data into google cloud storage from some source and download that into my local. I am able to do it and the job is also running fine, but my question is, is there is any possible way to check that all the data files from google cloud storage downloaded successfully?

As of now I have a below structure in my google cloud storage gs://bucket-name/some-folder/files.csv

I have multiple csv files in my storage bucket, and below is the command I'm using to download the data from cloud storage.

gsutil -m cp -R gs://bucket-name/some-folder/files.csv local_directory

Please let me know is there any possible way to ensure that my all files got downloaded successfully?

  • According to cloud.google.com/storage/docs/gsutil/addlhelp/… gsutil "...automatically performs integrity checks on all uploads and downloads." – kevlinux Apr 7 at 5:55
  • @kevlinux,but still if any one of the object fails while downloading then how can be know about the failure? – sam Apr 7 at 9:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.