cURL is a CLI tool for issuing HTTP GET, POST, & PUT request.

learn more… | top users | synonyms

1
vote
1answer
576 views

How do I create a local copy of a complete website section from OSX using curl?

I'm trying to download Mathematics of the Discrete Fourier Transform so that I can read through it off-line on my iPad (which doesn't have an Internet connection) I have used wget in Linux. It seems ...
5
votes
3answers
2k views

How do I use curl to download content from sourceforge?

Simply typing http://downloads.sourceforge.net/project/romfs/genromfs/0.5.2/genromfs-0.5.2.tar.gz works fine on a browser, but I'm trying to download from a CLI environment with limited utilities. ...
1
vote
1answer
230 views

What is wget idiom for curl -o?

So I use code in some function like if [ ! -e $DOWNLOADS/$1 ]; then echo_run curl -L http://$4/$5/$1 -o $DOWNLOADS/$1 fi I wonder how to achieve the same thing without curl?
5
votes
2answers
317 views

Make vim / curl recognize https:// links

I recently noticed vim/vimdiff can open links (using curl, it seems), which is an awesome feature. I wanted to use it today, like this: vimdiff deps ...
5
votes
4answers
3k views

Most straightforward way of getting a raw, unparsed HTTPS response

What would be the most straightforward way of making a GET request to a url over HTTPS, and getting the raw, unparsed response? Could this be achieved with curl? If so, what options would you need to ...
5
votes
3answers
770 views

Crontab job not emailing on failure

I have a cron job on Debian: 1 * * * * /home/paradroid/bin/myscript.sh >/dev/null There is an installed and configured MTA, and I have received emails when the script has a syntax error, so I ...
1
vote
1answer
137 views

Alert when a process exits

I often download files using wget and cURL. Is there anyway I can make my Linux box give a visual alert when either of those processes exit (i.e: the download completes) PS : I am using Ubuntu ...
5
votes
3answers
8k views

How do I get the http status of a site in bash?

I figure curl would do the job. I wrote in a bash script: #!/bin/sh function test { res=`curl -I $1 | grep HTTP/1.1 | awk {'print $2'}` if [ $res -ne 200 ] then echo "Error $res ...
2
votes
1answer
3k views

cURL/wget - SSLv3, TLS handshake, CERT hangs

I have problems connecting to https sites using cURL or wget. When I download from a https curl seems to be stuck while doing the TLS handshake, CERT. The problem is independent of the site (I noticed ...
6
votes
1answer
476 views

How to download all images whose size>100kb in a webpage?

I want to download images contained in some web pages, but only those that are larger than 100 KB. So far I've been downloading all the images and then deleting the small ones, but it's not very ...
4
votes
2answers
13k views

Set a network range in the no_proxy environment variable

I'm in a network using a proxy. I've got machines using lots of scripts here and there accessing each other over HTTP. The network is 10.0.0.0/8. My proxy is 10.1.1.1:81, so I set it up accordingly: ...
0
votes
1answer
377 views

Bash script doesn't TEE output to subdirectory

I've got a script that runs a number of curl commands by reading an input file line by line (each line has different curl parameters). I'm trying to tee the stnd out and stnd error of the curl ...
1
vote
1answer
625 views

cURL on Linux not compiled, although installed

cURL is not compiled into PHP. I'm running: [root@vps ~]# cat /etc/issue CentOS release 5.7 (Final) Kernel \r on an \m Then I execute yum install curl and yum install php-curl. Then: [root@vps ...
1
vote
1answer
650 views

Success/fail tests for cURL (don't execute second line if first line fails)

According to Using "&&" is not effective for cURL submit form (don't execute second line if first line fails) In below code; first line is curl submit; second line is for ...
2
votes
2answers
290 views

Using “&&” is not effective for cURL submit form (don't execute second line if first line fails)

I'm submitting to my website using cURL. I make cURL post it in timely fashion. In below code; first line is curl submit; second line is for queuing: curl -b cookies.txt \ -d title="$(sed ...
1
vote
1answer
428 views

Why does curl fail while using this export?

I'm doing some stuff with Facebooks FQL at the moment, and I'm using curl on the command line to test some of the responses of it. I'm also using export to keep the params seperated for easy editing. ...
2
votes
1answer
498 views

Asking curl to skip missing files

Is there any way to tell curl not to download a file when it does not exist on the server? I am using my Mac OS X shell to download a few files with curl. The command goes: curl -s -# ...
1
vote
1answer
723 views

rtorrent: Can't connect to trackers on CentOS 5.6

I'm trying to set up rtorrent on my CentOS 5.6 server, and it currently cannot connect to any trackers. I compiled libtorrent-0.12.6 and rtorrent-0.8.6. I also compiled libcurl 7.21.7 with c-ares ...
1
vote
3answers
3k views

Upgrading curl in Centos 5.6

There is any easy way to upgrade the default curl 7.15 to the 7.20 or 7.21 version? Any repository to install into YUM. Edit: I found this repository with the last version ...
1
vote
1answer
2k views

How can I specify that curl (via command line) overwrites a file if it already exists?

I'm using curl in this syntax: curl -o myfile.jpg http://mysite.com/myfile.jpg If I run this command twice, I get two files: myfile.jpg myfile-1.jpg How can I tell CURL that I want it to ...
2
votes
1answer
2k views

Install php_http on Arch Linux

First I installed the curl extension for PHP: $ sudo pacman -S php-curl and $ grep curl /etc/php/php.ini extension=curl.so Then I installed the PHP PEAR $ sudo pacman -S php-pear Having now ...
2
votes
2answers
467 views

Submit “&” to Drupal 6 form using cURL?

I'm using this code to create a node in Drupal 6: curl -b cookies.txt -d \ title="$(sed '1,/sblmtitle/d;/slpstitle/,$d' file.html)" \ -d form_build_id=form-5312da71abefb3bf1f63f1f2b7cf9105 \ ...
1
vote
1answer
497 views

Command-line cURL submit form from other files

In this command: curl -d param1=value1 -d param2=value2 http://example.com/submit If value1 is located in file1.txt, how can I make cURL include file1.txt in the command?
1
vote
1answer
522 views

submitting form with cURL sometimes works, sometimes doesn't

I use this command to fill out a form in Drupal 6 (create node): curl -b cookies.txt -d ...
2
votes
3answers
319 views

Source (.) piped curl output

I have a file, f, on my webserver with the following contents: alias ll='ls -l' I would like to (from another linux box) do the following, but I can't get the command right: curl http://myserver/f ...
0
votes
1answer
729 views

Curl and grep html text

Is there a way to grep html text from a curl request and output grab the value that is between two html tags? More specifically the input value of a form?
1
vote
1answer
463 views

Test FTP Username and Password

I'm working on a program that involves uploading some files via FTP. However, I'd like to abort the process if the username and password given by the user are incorrect. Is there any way to "test" if ...
2
votes
2answers
177 views

Is there a OS/user-wide log to log to?

I'm using curl to dowload a jar from a TESTDRIVE script and for debugging purposes it helps to log the http headers returned to a log file. But logging them in the working dir (where the script is ...
5
votes
2answers
5k views

loop curl get request bash

I'm trying to automate a curl script and eventually make it loop also. I found that I can't use any loop like: for i in `cat authors` do curl *line here* ...
7
votes
2answers
365 views

Are there any good tools besides SeleniumRC that can fetch webpages including content post-painted by JavaScript?

One major shortcoming of curl is that more and more wepages are having their main piece of content painted by a JavaScript AJAX response that occurs after the initial HTTP response. curl never picks ...
8
votes
3answers
2k views

Automating web requests using curl?

I have some "corporative" systems I have to use at work. Some of them are terrible, with frames, pop-ups and such. I was thinking about automating some stuff using curl or something like that. I need ...
2
votes
2answers
807 views

Puzzle: transfering files between two servers (ssh, ftp, http multipart…)

A puzzle: files being uploaded in realtime by SCP to SERVER1 need to be transfered to SERVER2. But: SERVER1 doesn't actually have network access to SERVER2 (NAT), however SERVER2 has SSH access to ...