If you already have a list of URLs, just add the POST data to that list. Something like:
Then, instead of using
-i, read the file in a bash loop and pass the data to wget:
while read url post; do wget --post-data="$post" $url; done < list.txt
To run them in parallel so that multiple files are downloaded at the same time, use
& instead of
;. Careful though, this will launch a separate
wget process for each URL.
while read url post; do wget --post-data="$post" $url & done < list.txt
One trick I use for launching this type of thing is keeping track of how many are currently running and only running the next one if the number is below a threshold, 10 for example:
while read url post; do
while [[ "$(pgrep -fc wget)" -gt 9 && $c -gt 10 ]]; do sleep 1; done;
echo "Launching $url ($c)";
wget --post-data="$post" $url >/dev/null 2>&1 && echo "$c finsihed" &
done < list.txt
That will launch the first 10 URLs, then wait for one to finish and launch the next.