0

I have a bash script that scrapes a list of urls for links to various kinds of documents. At the end, the script uses wget to download the files. However, I'm having trouble with filenames containing white space in the name: wget ends the url at the space. Is there some way to use sed or something to change the white space to %20 here? Or some other solution? This is my code:

for url in $(cat download.md)
do
    lynx --listonly --dump $url | \
awk '/\.(pdf|doc|docx|odt)$/{ sub("^[ ]*[0-9]+.[ ]*","",$0); print}'

done > ~/links.txt

for i in $( cat ~/links.txt ); do wget $i; 
done
6
  • I did, but it didn't work.
    – Linter
    Aug 29, 2018 at 18:52
  • wget "${i// /%20}"?
    – DopeGhoti
    Aug 29, 2018 at 18:52
  • @DopeGhoti also fails to download because the url is cut off at the space.
    – Linter
    Aug 29, 2018 at 18:58
  • Based on the for i in $( cat ... ) habit, I'd also recommend reading: unix.stackexchange.com/questions/169716/…
    – Jeff Schaller
    Aug 29, 2018 at 19:06
  • Thanks. I love how the example they laugh at is my actual code!
    – Linter
    Aug 29, 2018 at 19:09

1 Answer 1

1

You lost the full filenames during the $( ... ) command substitution. Just ask wget to read the file directly:

wget -i ~/links.txt
0

Not the answer you're looking for? Browse other questions tagged .