I am trying to download multiple files parallelly in bash and I came across GNU parallel. It looks very simple and straight forward. But I am having a hard time getting GNU parallel working. What am I doing wrong? Any pointers are appreciated. As you can see the output is very sequential and I expect output to be different each time. I saw a similar question in SO (GNU parallel not working at all) but that solutions mention there did not work for me.
svarkey@svarkey-Precision-5510:~$ seq 1 3 | xargs -I{} -n 1 -P 4 kubectl version --short=true --context cs-prod{} --v=6 I0904 11:33:10.635636 24861 loader.go:375] Config loaded from file: /home/svarkey/.kube/config I0904 11:33:10.640718 24863 loader.go:375] Config loaded from file: /home/svarkey/.kube/config I0904 11:33:10.640806 24862 loader.go:375] Config loaded from file: /home/svarkey/.kube/config I0904 11:33:11.727974 24863 round_trippers.go:443] GET https://kube-api.awsw3.cld.dtvops.net/version?timeout=32s 200 OK in 1086 milliseconds Client Version: v1.18.7 Server Version: v1.14.6 I0904 11:33:11.741985 24861 round_trippers.go:443] GET https://kube-api.awsw1.cld.dtvops.net/version?timeout=32s 200 OK in 1105 milliseconds Client Version: v1.18.7 Server Version: v1.14.6 I0904 11:33:11.859882 24862 round_trippers.go:443] GET https://kube-api.awsw2.cld.dtvops.net/version?timeout=32s 200 OK in 1218 milliseconds Client Version: v1.18.7 Server Version: v1.14.6 svarkey@svarkey-Precision-5510:~$ seq 1 3 | parallel -j 4 -I{} kubectl version --short=true --context cs-prod{} --v=6 Client Version: v1.18.7 Server Version: v1.14.6 I0904 11:33:18.584076 24923 loader.go:375] Config loaded from file: /home/svarkey/.kube/config I0904 11:33:19.662197 24923 round_trippers.go:443] GET https://kube-api.awsw1.cld.dtvops.net/version?timeout=32s 200 OK in 1077 milliseconds Client Version: v1.18.7 Server Version: v1.14.6 I0904 11:33:18.591033 24928 loader.go:375] Config loaded from file: /home/svarkey/.kube/config I0904 11:33:19.691343 24928 round_trippers.go:443] GET https://kube-api.awsw3.cld.dtvops.net/version?timeout=32s 200 OK in 1099 milliseconds Client Version: v1.18.7 Server Version: v1.14.6 I0904 11:33:18.591033 24924 loader.go:375] Config loaded from file: /home/svarkey/.kube/config I0904 11:33:19.775152 24924 round_trippers.go:443] GET https://kube-api.awsw2.cld.dtvops.net/version?timeout=32s 200 OK in 1183 milliseconds svarkey@svarkey-Precision-5510:/tmp/parallel-20200822$ parallel --version GNU parallel 20200822 Copyright (C) 2007-2020 Ole Tange, http://ole.tange.dk and Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. GNU parallel comes with no warranty. Web site: https://www.gnu.org/software/parallel
echo
calls are probably just pretty fast ...parallel
output is sequential because it captures the processes output and prints it only when that process is finished, unlikexargs
which let the processes print output immediately.sleep
.