I'm using gnu parallel to generate backups for about ~1500 web sites on pantheon.io, using their terminus CLI. The terminus backup:create
command does not finish until a response is received that it has completed on the remote end. I'm wondering if there is any way to better speed this up with parallel so that more sites can be backing up while waiting on previous ones to complete or just run more overall if not. If it makes any difference, this is being run from a Jenkins CI job. Thank you.
#!/bin/bash +x
backup_sites() {
BACKUP=$(terminus backup:create "$*".live)
echo "$*": "$BACKUP"
}
SITE_LIST=$(terminus site:list --field=name)
export -f backup_sites
echo "$SITE_LIST" | parallel backup_sites