cURL is a CLI tool for issuing HTTP GET, POST, & PUT request.

learn more… | top users | synonyms

8
votes
3answers
2k views

Automating web requests using curl?

I have some "corporative" systems I have to use at work. Some of them are terrible, with frames, pop-ups and such. I was thinking about automating some stuff using curl or something like that. I need ...
7
votes
2answers
431 views

Are there any good tools besides SeleniumRC that can fetch webpages including content post-painted by JavaScript?

One major shortcoming of curl is that more and more wepages are having their main piece of content painted by a JavaScript AJAX response that occurs after the initial HTTP response. curl never picks ...
17
votes
3answers
6k views

why would curl and wget result in a 403 forbidden?

I try to download a file with wget and curl and it is rejected with a 403 error (forbidden). I can view the file using the web browser on the same machine. I try again with my browser's user agent, ...
6
votes
3answers
1k views

How to tweet using terminal?

I would like to tweet a message using terminal. I tried something like: curl -u 'TwitterUsername':'TwitterPassword' -d status=”Your Message Here” https://twitter.com/statuses/update.xml but seems ...
5
votes
2answers
6k views

loop curl get request bash

I'm trying to automate a curl script and eventually make it loop also. I found that I can't use any loop like: for i in `cat authors` do curl *line here* ...
1
vote
1answer
1k views

How to delete the oldest file over FTP using CURL?

I am trying to write a script that will use CURL to FTP to a folder on a file server, then count the files in that folder. If there are more than 5 files, delete the oldest one. I know how to delete ...
7
votes
6answers
2k views

Is there a way to use curl interactively? Or is there an interactive curl/wget shell?

Imagine something like this: $ curlsh http://www.example.org > GET /foo/bar/bam ...output here... > POST /thing/pool ... ... result here.... is there a tool that lets me do that?
23
votes
4answers
10k views

Throttle the download speed of wget or curl while downloading

Is it possible to throttle (limit) the download speed of wget or curl ? Is it possible to change the throttle value while it is downloading ?
6
votes
3answers
4k views

How do I use curl to download content from sourceforge?

Simply typing http://downloads.sourceforge.net/project/romfs/genromfs/0.5.2/genromfs-0.5.2.tar.gz works fine on a browser, but I'm trying to download from a CLI environment with limited utilities. ...
7
votes
5answers
1k views

Read and confirm shell script before piping from curl to sh (curl -s [url] | sh)

Whenever I have to execute a shell script from the web curl -s [url] | sh, I first open url in my web browser to make sure the script isn't malicious and is safe to run. I remember seeing a command ...
6
votes
2answers
1k views

Pipe Password to Application When Asked

The following should be done in a bash script: curl --digest --user schmijos https://bitbucket.org/u/p/get/tip.zip -o tip.zip How can I automatically submit a password to curl when it asks for it? ...
2
votes
1answer
275 views

How to download .pdf files from pubmed

I can't download .pdf files from http://www.ncbi.nlm.nih.gov/pmc/articles, I already have some document IDs such as PMC3404089. I tried to issue the following command: curl -o file.pdf ...
2
votes
1answer
535 views

Strange issue with curl/lynx and non resolving domains

I have a weird issue on my server in that if I use curl or lynx to try and access a URL that doesn't exist (say a non registered domain, or a domain that's not in DNS), I don't get an error, it ...
2
votes
2answers
295 views

Using “&&” is not effective for cURL submit form (don't execute second line if first line fails)

I'm submitting to my website using cURL. I make cURL post it in timely fashion. In below code; first line is curl submit; second line is for queuing: curl -b cookies.txt \ -d title="$(sed ...
1
vote
1answer
620 views

How do I create a local copy of a complete website section from OSX using curl?

I'm trying to download Mathematics of the Discrete Fourier Transform so that I can read through it off-line on my iPad (which doesn't have an Internet connection) I have used wget in Linux. It seems ...
1
vote
1answer
148 views

Alert when a process exits

I often download files using wget and cURL. Is there anyway I can make my Linux box give a visual alert when either of those processes exit (i.e: the download completes) PS : I am using Ubuntu ...
1
vote
1answer
579 views

submitting form with cURL sometimes works, sometimes doesn't

I use this command to fill out a form in Drupal 6 (create node): curl -b cookies.txt -d ...