cURL is a CLI tool for issuing HTTP GET, POST, & PUT request.

learn more… | top users | synonyms

2
votes
3answers
158 views

How to download files without full URL?

I was always wondering how to download the files through the Linux shell (I have wget, curl) that do not have a full URL of the file to be downloaded, but the full URL is passed e.g. to the browser ...
2
votes
3answers
199 views

curl page source from text file containing URLs

I have a text file containing up to 100 URLs. I am able to curl the page source from them using: cat /path/to/url.txt|xargs curl -o /path/to/output.txt This will download the page source for all of ...
1
vote
2answers
119 views

(Shell Script) Variable not pass correctly when fetching value from file?

I have this section of curl command inside my shell script which caused a error 500 repsonse from the server. However, this code works when I manually run the command. curl -qgsSkH "Content-Type: ...
0
votes
2answers
142 views

How to get the contents of a webpage but after running javascript?

I am trying to download the contents of a web page using curl but it seems l am missing some content that l suspect to be generated by JavaScript. If the page runs on the browser l am able to get all ...
2
votes
2answers
68 views

Extract info with curl or something

I plan to write a script for automatic VirtualBox build I see every release VirtualBox put a version (of course) and a release version. In this example ...
0
votes
1answer
26 views

Using xargs to generate links

I have a file that looks like this: abc,123,x xabc,345,xy cabc,6234,ab abgc,12,yo jabc,9991,to I need to generate a list of links: http://www.mywebsite.com?param1=abc&param2=123&param3=x ...
1
vote
1answer
699 views

Use curl to download list of files in csv

I have a csv of JPG url extensions. http://www.example.com/images/[url_extension] I want to use curl to loop through the CSV and download the jpg at each extension. So far I have the following, but ...
1
vote
0answers
181 views

TOR hidden service not always accessible through cURL. Takes multiple tries

When I try to access a hidden service on TOR using cURL, for some reason I'm not getting access to the site 100% of the time. Many times it returns "curl: (7) Can't complete SOCKS5 connection to ...
8
votes
2answers
319 views

What is most secure and simplest way to have a user-typed password on bash become part of stdin to a program?

I'm looking for the (1) most secure and (2) simplest way to have a user type a password on a bash shell prompt and to have that password become part of stdin to a program. This is what the stdin ...
1
vote
1answer
102 views

Pipe multiple file from curl to other program

I would like to download several images (from a server with a self-signed certificate) and view them all with feh. Is this possible with a single command? Something like curl -sk -b ...
0
votes
1answer
199 views

curl and wget cut off url

I have a web cms that allows so called action IDs which are urls. When you hit those, they perform some code in the cms. So I wanted to perform an action periodically and used cron for that. PROBLEM: ...
0
votes
3answers
79 views

Make curl/cat not scroll down when given long text?

I have made a program where I read the text from a url using curl. The problem is that when given texts that are larger than the terminal window, it automatically scrolls down to the bottom, I want to ...
1
vote
1answer
103 views

curl progress in dialog

how can i properly display the curl progress in the dialog window? curl http://mysite.corp/image/root_21.tar.bz2 | tar -C /mnt/dest/ -jxf - i tried this command but as you can see it does not ...
-2
votes
1answer
124 views

Shell script curl limit the try time for page not found

I found following code to check web url. if curl --output /dev/null --silent --head --fail "$url"; then echo "URL exists: $url" else echo "URL does not exist: $url" fi Here if the page not ...
1
vote
1answer
81 views

Using curl as a test [duplicate]

I'm writing Puppet configuration to automate the creation of an Elastic Search repository resource. Unfortunately, as far as I can tell, there's no way to specify this configuration in the Elastic ...
0
votes
1answer
685 views

Script with curl works manually but not in cron job

My script (status.sh) is: #!/bin/bash SITE=http://www.example.org/ STATUS=$(/usr/bin/curl -s -o /dev/null -I -w "%{http_code}" $SITE) if [ $STATUS -eq 200 ] then echo $STATUS >> ...
0
votes
1answer
41 views

Trying to request a variable before running a curl and read back as integer

echo -e "Enter layoutID: \c" read layoutID echo -e "Enter playlistID: \c" read playlistID echo -e "Enter siteID: \c" read siteID curl -H "Content-Type: application/json" -d ...
-1
votes
2answers
110 views

Curl, ping hangs/waits when using private network ips

I have a whole bunch of servers working inside my office premises private network. When I try to run my tests outside office network, curl/ping/python/nodejs requests library, all of them wait ...
1
vote
0answers
190 views

Using curl results in error: “Library not loaded: /usr/local/lib/libcurl.4.dylib …”

I had a few issues with the cURL library interfering with Homebrew. After hopefully removing the library (I no longer use it anyway), I started receiving errors whenever I used commands such as curl ...
3
votes
2answers
319 views

Embed “cat file” in CURL data

I would like to cat the contents of a file.txt and use this as part of the --data field in my curl query. I tried: curl ... --data '{ "file_content":[" $(cat file.txt) ", ...]}"' ... but that ...
1
vote
2answers
848 views

Execute CURL command only once in script?

To explain a little, I am currently monitoring a folder for any changes and when changes are detected it simply uploads the detected files to my server via rsync. This is working without any issues : ...
0
votes
3answers
194 views

Curl freezes when downloading from ftp

I'm trying to download a file from ftp server using curl: curl --user kshitiz:pAssword ftp://@11.111.11.11/myfile.txt -o /tmp/myfile.txt -v curl connects to the server and freezes: * Hostname was ...
1
vote
1answer
153 views

How do I display all URLs in a redirect chain?

I'm looking for a way to show all of the URLs in a redirect chain, preferably simply and from the shell. I've found a way to almost do it with curl, but it only shows the first and last URL. I'd like ...
1
vote
3answers
710 views

echo text after curl command on same line

#!/bin/bash echo "$(curl -s -I https://google.com|grep Server)" abc returns: abcer: gws and bash -x shows: ++ grep Server ++ curl -s -I https://google.com ' abco 'Server: gws abcer: gws but ...
1
vote
1answer
180 views

cURL download with multiple ranges

I'm trying to download a series of podcast episodes using cURL. I know how to use a range of numbers to download several files matching a pattern, but what I can't figure out is how to have multiple ...
0
votes
1answer
657 views

“dyld: Library not loaded: /usr/local/lib/libcurl.4.dylib” while executing curl with a unprivileged user

My MAC OS is Yosemite 10.10, which has an old version of curl. I have installed Xcode and Xcode command line tools. I run cURL with the "sudo" command as “sudo curl”: sudo curl Password: ...
0
votes
1answer
1k views

curl script - Wait for download to finish

Below is part of a script I use to download files from a website daily. However recently they added speed limits to downloading files. I increased the sleep time but then everything else takes too ...
2
votes
1answer
536 views

“Network is unreachable”, github, bitbucket is not working

The output of ifconfig eth0 Link encap:Ethernet HWaddr 3c:97:0e:74:43:8e UP BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 ...
1
vote
1answer
36 views

wget a url from my public outdoor wx cam

I am trying to grab an image from my weathercam using wget and cron. For now I just need to see how to get an image downloaded and timestamp the filename. ...
0
votes
1answer
257 views

Using curl with xargs connection hanging sometimes - any way to retry?

echo $LIST | xargs -I{} -n 1 -P 8 sh -c 'curl -o backup.dump http://url.com' I am running this for a large amount of files (around 300). After a while, sometimes curl hangs and won't progress for a ...
1
vote
1answer
178 views

Crawling generated source of page

I am trying to crawl the source of this website of a research article, exploiting a redirect from it's Document Object Identifier (DOI): doi=10.1006/jfin.1996.0208 url=$(curl -L -s -w ...
0
votes
0answers
235 views

Need to test web service using curl or wGet in Shell Script?

I use SoapUI to verify my responses below are the sample details. I am planning to automate the processes in linux. How can I achieve it using curl? or wget? I would like to receive the result in a ...
-1
votes
1answer
18 views

Need to write an automated web navigator with “click” and “retrieve” capabilities

In our company, several clients provide the detailed information information that we need in a web site. The amount of time that our employees spend navigating the remote website is simply ridiculous, ...
1
vote
2answers
204 views

Equal command between wget and cURL (with options)?

I usually use wget to download remote file, the command I use: wget --timeout=5 --tries=1 --referer=http://www.example.com 'http://www.example.com/abcdefg123.jpg' -O image.jpg As you can see, there ...
3
votes
2answers
4k views

How to convert ssl ciphers to curl format?

The official ssl docs list ciphers in a different format than curl takes. For instance, if I want curl to use the cipher TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, I have to pass it curl --ciphers ...
3
votes
1answer
2k views

How to see list of curl ciphers?

My understanding is that during ssl negotiation, the client (i.e. curl) sends a list of ciphers to the server, and the server replies with its preferred choice. How do I see the list of ciphers that ...
0
votes
0answers
280 views

CURLE_SSL_CACERT_BADFILE cURL error (77)

I have getting the cURL error CURLE_SSL_CACERT_BADFILE on my Nginx webserver. I found a fix where I add my pem link to my curl.ini file. Before I did this I would get the error on curl ...
2
votes
1answer
242 views

What data transformations does curl's --data option perform?

I recently used curl's --data option to upload some JSON to a server, and after reading the manpage, I'm not sure whether I can have faith that it got there intact. curl's manpage explicitly ...
2
votes
1answer
79 views

Can you fetch git commit data using cURL?

I need to fetch the object of an annotated tag (i.e. the annotation message) from a remote repository and so far there doesn't seem to be a command for that (i.e. ls-remote provides only the refs but ...
-1
votes
1answer
221 views

Webpage content - search for pattern & save to file

I have a web page, say https://test.salesforce.com. I would like to search for a pattern, e.g. "src" at the beginning of a line in the entire webpage content. If the pattern is found, I need to ...
0
votes
1answer
584 views

Send email depending on success of curl command

I created a script to monitor and have a query as below :- httpUrl="http://some.website.ashx" balExp=$(curl -v -X POST -d "UID=test&PWD=2222" $httpUrl) How do I sent email alert (my email for ...
1
vote
0answers
89 views

setting up pycurl for embedded ARM

I am working on a linux x86_64 machine, and I am trying to set up pycurl for an embedded ARM processor running petalinux. I need pycurl library before building the embedded linux image. I have ...
8
votes
1answer
669 views

How can I login to a website from terminal?

As a student of computer science, I am now learning to work with command line. I need to use brute force to login to a website, knowing the username and that the password is an integer in a certain ...
0
votes
0answers
389 views

Using brew: curl version error, “Incompatible library version” for libcurl.4.dylib

I used to have a working version of homebrew but then something broke today related to curl and libcurl.4.dylib— possibly a version or file location issue. I'm on Mac OS X 10.9.5 (Mavericks). For ...
0
votes
2answers
486 views

Download an article with cURL given a dynamic download link

I'm trying to download this published journal article using cURL. It's the main page of an open access, so there should be not problems for anyone to see/download the article. I then extract the ...
10
votes
1answer
15k views

Hide curl output [duplicate]

I'm making a curl request where it displays an html output in the console like this <b>Warning</b>: Cannot modify header information - headers already sent by (output started at ...
0
votes
1answer
78 views

Nnumber of HTML redirects from cURL

My little script extracts DOI's from a csv files in order to check whether they are right. The number should equal 3, and if it doesn't, I want to output an message. #!/bin/bash ...
-1
votes
1answer
133 views

Accessing array elements within process substitution bash

I've got a text file which looks like this: b4238ca2-cb8d-11e4-8731-1681e6b88ec1,https,username,password,ipaddress b4238f0e-cb8d-11e4-8731-1681e6b88ec1,https,username,password,ipaddress ...
0
votes
2answers
546 views

Yum Broke For some reason

i am trying to install Mysql Client i am getting this annoying message. I am still unable to find the proper answer. [root@localhost Downloads]# yum install mysql-bench There was a problem importing ...
1
vote
2answers
305 views

How do I post something using cURL that starts with `@`?

The command curl http://localhost/ --data @hello will try to read from file hello. How do I escape the @ symbol?