New answers tagged

0

You can use -o log for that, then navigate and extract links from the log file using this https://www.garron.me/en/bits/wget-download-list-url-file.html .-


1

Since OC 9.0(?) Filesystem checks are disabled by default for more performance. Check and maybe change your config.php to 'filesystem_check_changes' => 1, https://doc.owncloud.com/server/7.0EE/admin_manual/configuration_server/config_sample_php_parameters.html


0

ok I finally sat down and looked into the contents of the file. It appears that kickass torrents sends a gziped version of the torrent file and the browser quietly gunzips it. so I created the following download script to get the torrent files. #!/bin/bash torr_link="${1}" #perl parses the link and keeps as torrent title whatever comes after the variable ...


0

I fought with this today aswell. In my case the problem was with german letters like "ä,ö,ü"... I fixed it by setting ALL my language settings to UTF-8. You can see a tutorial here: https://perlgeek.de/en/article/set-up-a-clean-utf8-environment


1

PHP has a class DOMDocument that you can use to retrieve and parse html. this code will fetch and extract the rows from the webpage. There is still more work necessary to extract the specific items you want but if you are willing to learn some PHP this will get you started <?php $html = file_get_contents('http://currency.poe.trade/search?league=...


0

Tried all of the above however no luck; used dev browser tool to get user-agent string, once I added the following, success: --user-agent="Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"


0

Using awk: awk '{split($2, p, "."); system(sprintf("wget -O %s.%s %s\n", $1, p[length(p)], $2))}' images.txt Break down First we use awk to print out roughly the command we want: awk '{printf("wget -O %s %s\n", $1, $2)}' images.txt Then we extract the extension by splitting the URL on '.' and append it to the first argument: awk '{split($2, p, "."); ...


3

This might do your job, while read a b do wget "$b" -O "$a".jpg printf "$a".jpg"\n%s" >> newfile done < images.txt


2

To test: $ awk '{ ext=gensub(/.*(\.[^.]*$)/,"\\1",1); print "wget " $2 " -O " $1 ext}' images.txt wget image.jpg -O some_id1.jpg wget image2.jpg -O some_id2.jpg wget image3.jpg -O some_id3.jpg To run, pipe into bash or sh like so: awk '{ ext=gensub(/.*(\.[^.]*$)/,"\\1",1); print "wget " $2 " -O " $1 ext}' images.txt | bash Add single-...


1

wget -nd -r -l1 -P /save/location -A jpeg,jpg http://www.example.com/products Explanation : -nd prevents the creation of a directory hierarchy (i.e. no directories). -r enables recursive retrieval. See Recursive Download for more information. -l1 Specify recursion maximum depth level. 1 for just this directory in your case it's products. -P sets the ...


1

Try httrack(1), a web spider that is most useful for creating local mirrors of entire web sites. Homepage: https://www.httrack.com/ Manpage: http://www.httrack.com/html/httrack.man.html The examples in the linked manpage should get you started.


1

Seems like it is not trivial to get directory listing over http; I could get the bz2 files using bellow: wget -k -l 0 "http://archive.xfce.org/xfce/4.6.2/src/" -O index.html ; cat index.html | grep -o 'http://archive.xfce.org/xfce/4.6.2/src/[^"]*.bz2' | uniq -c | xargs wget Hope this help.



Top 50 recent answers are included