cURL is a CLI tool for issuing HTTP GET, POST, & PUT request.

learn more… | top users | synonyms

9
votes
3answers
4k views

Automating web requests using curl?

I have some "corporative" systems I have to use at work. Some of them are terrible, with frames, pop-ups and such. I was thinking about automating some stuff using curl or something like that. I need ...
8
votes
2answers
554 views

Are there any good tools besides SeleniumRC that can fetch webpages including content post-painted by JavaScript?

One major shortcoming of curl is that more and more wepages are having their main piece of content painted by a JavaScript AJAX response that occurs after the initial HTTP response. curl never picks ...
23
votes
3answers
16k views

why would curl and wget result in a 403 forbidden?

I try to download a file with wget and curl and it is rejected with a 403 error (forbidden). I can view the file using the web browser on the same machine. I try again with my browser's user agent, ...
2
votes
2answers
861 views

Can't install cURL on Linux Mint

I am trying to install cURL in order to be able to install the Ruby version manager. I write this: sudo apt-get update && apt-get install curl And get this output: E: Some index files ...
1
vote
2answers
2k views

trying to install curl with homebrew, getting “incompatible library version” for libcurl.4.dylib

I am on OSX 10.9.5. I would like to use a later version of CURL than what is bundled with OSX, but I do not want to remove the OSX binary just in case, so I wanted to opt to install it via homebrew. ...
9
votes
3answers
10k views

Get URL's redirect target with curl

I would like to check where a single URL redirects. An example of that could be a link from Google's search result page (where a click always goes through Google server). Can I do that with curl?
6
votes
3answers
2k views

How to tweet using terminal?

I would like to tweet a message using terminal. I tried something like: curl -u 'TwitterUsername':'TwitterPassword' -d status=”Your Message Here” https://twitter.com/statuses/update.xml but seems ...
5
votes
2answers
7k views

loop curl get request bash

I'm trying to automate a curl script and eventually make it loop also. I found that I can't use any loop like: for i in `cat authors` do curl *line here* ...
2
votes
3answers
14k views

Health check of web page using curl

I'd like to do a health check of a service by calling a specific url on it. Feels like the simplest solution would be to use cron to do the check every minute or so. In case of errors, cron sends me ...
1
vote
1answer
149 views

Using cURL command-line tool on Mac, how does one fetch stock data which comes back *without* commas? [duplicate]

The below script is not working. cd desktop/quoteUpdate while true do curl-o quotes.txt -s "http://download.finance.yahoo.com/d/quotes.csv?s=goog,aapl&f=sl1" sed -i '.bak' 's/,/ /g' quotes.txt ...
1
vote
1answer
2k views

How to delete the oldest file over FTP using CURL?

I am trying to write a script that will use CURL to FTP to a folder on a file server, then count the files in that folder. If there are more than 5 files, delete the oldest one. I know how to delete ...
0
votes
1answer
303 views

Using the cURL command, how does one fetch stock data which comes back *without* commas and spaces?

What can I add to the below script so that it fetches the stock data and brings it back WITHOUT any commas and spaces? For example GOOG Outstanding shares is 675,000,000. I desire the output of ...
7
votes
6answers
2k views

Is there a way to use curl interactively? Or is there an interactive curl/wget shell?

Imagine something like this: $ curlsh http://www.example.org > GET /foo/bar/bam ...output here... > POST /thing/pool ... ... result here.... is there a tool that lets me do that?
36
votes
4answers
17k views

Throttle the download speed of wget or curl while downloading

Is it possible to throttle (limit) the download speed of wget or curl ? Is it possible to change the throttle value while it is downloading ?
8
votes
3answers
6k views

How do I use curl to download content from sourceforge?

Simply typing http://downloads.sourceforge.net/project/romfs/genromfs/0.5.2/genromfs-0.5.2.tar.gz works fine on a browser, but I'm trying to download from a CLI environment with limited utilities. ...
8
votes
5answers
2k views

Read and confirm shell script before piping from curl to sh (curl -s [url] | sh)

Whenever I have to execute a shell script from the web curl -s [url] | sh, I first open url in my web browser to make sure the script isn't malicious and is safe to run. I remember seeing a command ...
2
votes
1answer
731 views

Strange issue with curl/lynx and non resolving domains

I have a weird issue on my server in that if I use curl or lynx to try and access a URL that doesn't exist (say a non registered domain, or a domain that's not in DNS), I don't get an error, it ...
6
votes
2answers
2k views

Pipe Password to Application When Asked

The following should be done in a bash script: curl --digest --user schmijos https://bitbucket.org/u/p/get/tip.zip -o tip.zip How can I automatically submit a password to curl when it asks for it? ...
2
votes
1answer
673 views

How to download .pdf files from pubmed

I can't download .pdf files from http://www.ncbi.nlm.nih.gov/pmc/articles, I already have some document IDs such as PMC3404089. I tried to issue the following command: curl -o file.pdf ...
2
votes
2answers
305 views

Using “&&” is not effective for cURL submit form (don't execute second line if first line fails)

I'm submitting to my website using cURL. I make cURL post it in timely fashion. In below code; first line is curl submit; second line is for queuing: curl -b cookies.txt \ -d title="$(sed ...
1
vote
1answer
645 views

How do I create a local copy of a complete website section from OSX using curl?

I'm trying to download Mathematics of the Discrete Fourier Transform so that I can read through it off-line on my iPad (which doesn't have an Internet connection) I have used wget in Linux. It seems ...
1
vote
1answer
174 views

Alert when a process exits

I often download files using wget and cURL. Is there anyway I can make my Linux box give a visual alert when either of those processes exit (i.e: the download completes) PS : I am using Ubuntu ...
1
vote
1answer
655 views

submitting form with cURL sometimes works, sometimes doesn't

I use this command to fill out a form in Drupal 6 (create node): curl -b cookies.txt -d ...
0
votes
1answer
146 views

“dyld: Library not loaded: /usr/local/lib/libcurl.4.dylib” while executing curl with a unprivileged user

My MAC OS is Yosemite 10.10, which has an old version of curl. I have installed Xcode and Xcode command line tools. I run cURL with the "sudo" command as “sudo curl”: sudo curl Password: ...