New answers tagged

1

Here's an improved script: #!/bin/sh cat file.txt |while read url do lynx -listonly -dump "$url" done | awk '/^[ ]*[1-9][0-9]*\./{sub("^ [^.]*.[ ]*","",$0); print;}'| \ sort -u allowing for any type of URL recognized by lynx (including ftp for instance). The script sorts the result, eliminates duplicates (which lynx will not do by itself). Further ...


0

Calling list.txt your list: for i in $(cat list.txt) do lynx -accept_all_cookies -dump $i |grep "http" |sed -e "s/^.*http/http/" done I suggest to redirect the output on some file.



Top 50 recent answers are included