Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.
Sign up to join this community
I have a ~100 .tar files stored on AWS EFS, each one is ~6GB in size.
I have mounted the EFS to my EC2 machine, looped over each .tar file, and untarred it into EBS.
The thing is - that it takes quite some time.
I am running on a 1.2TB EBS with 3000 iops set, untarring using pigz - any optimization tips?
Open both EFS and EBS monitoring tabs (or CloudWatch for greater details) and find out whether the bottleneck is EFS read performance or EBS write performance. It could even be the CPU - check the monitoring there too, incl CPU credits if you are on t2/t3 instances.
Once you know where the bottleneck is do something about it - add more EFS bandwidth, change instance type, etc.
Hope that helps :)
