I would like to only copy files from S3 that are from today out of a certain bucket with 100s of files. I tried the following: $ aws s3 ls s3://cve-etherwan/ --recursive --region=us-west-2 | grep 2018-11-06 | awk '{system("aws s3 sync s3://cve-etherwan/$4 . --region=us-west-2") }' but it doesn't quite work, I also get files from other dates. How do I do this correctly?


That's because in the second aws s3 you use sync. Try cp instead. Also you can merge the "grep" and "awk" together.

$ aws s3 ls s3://cve-etherwan/  --recursive | awk '/^2018-11-06/{system("aws s3 cp s3://cve-etherwan/$4 .") }'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.