1
vote
1answer
208 views

How to implement retry mechanism while executing the server url in the shell script?

I am working on a project in which I need to make a url call to one of my server from the bash shell script.. http://hostname.domain.com:8080/beat After hitting the above url, I will be getting the ...
1
vote
1answer
434 views

syntax error in conditional expression while executing the shell script?

I am working on a project in which I need to make a url call to one of my server from the bash shell script.. http://hostname.domain.com:8080/beat After hitting the above url, I will be getting the ...
0
votes
2answers
124 views

Simplest wget example Scientific Linux

I'm on Sci Linux and know nothing about commands. How can I use the wget command? Can someone provide a simple example where it actually downloads something? How do you specify where the download ...
2
votes
4answers
469 views

Using wget and grep to download HTML pages and filter by keywords

KEYWORDS=("dolphins" "fish" "dogs" "cats" "iguanas") IFS=$'\n' find . -type f -size +1c ! -exec grep -qF "${KEYWORDS[*]}" {} \; -exec truncate -s 0 -f {} \; If a file is over 1 byte, search for ...
2
votes
2answers
2k views

Running thousands of curl background processes in parallel in bash script

I am running thounsand of curl background processes in parallel in the following bash script START=$(date +%s) for i in {1..100000} do curl -s "http://some_url_here/"$i > $i.txt& ...
1
vote
1answer
664 views

How can I use wget to download large files? [closed]

I want to get the ISO for a particular version of Linux, say 'Mint 14 “Nadia” KDE '. I tried using wget in the terminal to download the ISO of this distro: $ wget ...
6
votes
1answer
6k views

How to get text of a page using wget without html?

If I try wget on a webpage, I am getting the page as html. Is it possible to retrieve only text of a file without associated html ? (This is required for me since some of the HTML pages contains c ...
1
vote
1answer
1k views

How to know whether wget has completed download successfully?

I attempted to download openSUSE 12.1 through wget. It stopped after 64% showing: 05:54:48 (120.30 KB/s) - `openSUSE-12.1-DVD-i586.iso' saved 4429185024/4429185024] It appears that the download is ...
1
vote
1answer
1k views

how to specify the forwarding port when using multiple tsocks services?

According to the answer of this question, I have my /etc/tsocks.conf containing these lines: path { server = localhost server_port = 1081 reaches = <ip-address-of-server-b>/32 } path { server = ...
4
votes
2answers
805 views

Does -q definitely turn off wget output logging?

I currently have a shell script running on a linux server which is using wget in oder to download a remote web page. This in turn is executed by a cron job which is scheduled to run at certain times. ...
2
votes
1answer
1k views

wget -c website's certificate verification

anisha@linux-dopx:~> wget -c --no-check-certificate --user abc --password xyz https://www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf --2011-08-29 15:11:53-- ...
1
vote
3answers
581 views

wget batch linux file

I am trying to download images from a manufacturer's site using wget. When I run the command from the shell it works fine. But when I run it from a file I am getting a placeholder file. Here is the ...