Hot answers tagged

8

I did some performance tests with wget and curl, and the result is: 100 times tested average run time while download 1MB file: wget: 0.844s cURL: 0.680s 100 times tested average run time while download 5MB file: wget: 1.075s cURL: 0.863s 100 times tested average run time while download 10MB file: wget: 1.182s cURL: 1.074s Command size on the system: ...


7

Curl is failing because that site is incorrectly configured Certificates are used to sign other certificates, forming chains. A CA has a root certificate, which is trusted by operating systems and browsers. This root certificate is most commonly used to sign one or several intermediate certificates, which in turn are used to sign leaf certificates (that can ...


7

The -i option means curl will include http response headers which are not in JSON format. That is what causes your parse error, however given the json you provided, you need to use [] to tell it to iterate the array: curl 'https://api.shodan.io/shodan/alert/info?key={API KEY}' | jq '.[].id' alternatively (and IMO more intuitively) using json: curl 'https:...


6

You can check the installed package version with dpkg -l | grep -i openssl. The actual TLS/SSL version used depends on what the server offers or what is negotiated between server and client in any given session.


6

That URL contains an &, this means that your command would start curl http://example.com/endpoint/?arg1=val1 in the background, and then set the shell variable arg2 to the value val2, since that's what the next bit of the URL means when the shell parses the line. Use quotes around the URL: curl 'http://example.com/endpoint/?arg1=val1&arg2=val2'


5

UPDATE As @marcelm pointed out, the answer below is not correct. [you could say it's dirty walkaround of a problem] The [discovery.ucl.ac.uk] server was not sending intermediate certificate authority, which resulted in a incomplete certification chaine. By my mistake, answer below is to download the missing intermediate certificate authority, not root ca [...


5

The curl binary that's built for Debian and Ubuntu doesn't contain support for libc-ares. That's because that library can't return both A (IPv4) and AAAA (IPv6) queries in a single response, so if the host you're connecting to uses IPv6 but you don't have IPv6 support, the connection just fails. Since this is a pretty big defect (since many open Wi-Fi ...


4

It understands it alright, but not as you would wish. The regular expression fragment {* matches zero or more { characters and n* matches zero or more n characters, so the full expression {*clipper-coin*} would match {clipper-coin}, {{{clipper-coi}, or clipper-coinnnnnn} etc. Line-oriented text-processing utilities such as most of the utilities in the ...


4

Checking the script file, you can use the -y option: % sh <(curl https://sh.rustup.rs -sSf) -h rustup-init 1.18.3 (302899482 2019-05-22) The installer for rustup USAGE: rustup-init [FLAGS] [OPTIONS] FLAGS: -v, --verbose Enable verbose output -y Disable confirmation prompt. --no-modify-path Don't ...


4

The script that you fetch using curl --silent and then execute with bash contains a call to curl without the --silent option, which is why you're seeing the progress indicator. The relevant part of the remote script: curl -H 'Cache-Control: no-cache' "https://raw.githubusercontent.com/oresoftware/run-tsc-if/master/run.sh?$(date +%s)" \ --output "$HOME/....


4

Install Ondrej Sury's repo, I successfully installed php7.3-curl from it. It seems that the version it proposes for php7.2-curl depends on libcurl4 on Debian 10, which would make it work. To install the repo for Debian 10, add deb https://packages.sury.org/php/ buster main to your /etc/apt/sources.list, or to a new php.list file under /etc/apt/sources....


4

If you want to install PHP 7.2, you’ll probably have to revert to Debian 9, yes. The libcurl transition is a rather complex one and it’s impossible to mix and match libcurl3 and libcurl4-based packages; since Debian 10 uses libcurl4, you need to use packages depending on that. (Ondrej may well provide Debian 10-compatible PHP 7.2 packages at some point, I ...


4

curl's -T accepts - to read from stdin. The command reads like: cat test.txt | curl -T - ftp://admin:PASS@192.168.1.10/file.txt Here's an example used to upload a file to Vimeo directly: gsutil cp gs://my-bucket/video.mp4 - | curl -T - ftp://user:PASS@ftp-3.cloud.vimeo.com/video.mp4


4

apt accepts multiple packages to install, this is what the ... from man apt means: apt install pkg... Your command will install the packages curl, nano and unzip. All questions (e.g. Do you want to install ...) will be answered with yes (-y). Generally, if you want to understand commands, you should check the synopsis from the commands help or man page ...


4

curl 'https://api.ipify.org?format=json' | jq -r '.ip' This would use jq to extract the value associated with the top-level ip key in the JSON response from curl. You could then use this to make your other curl call: ipaddr=$( curl 'https://api.ipify.org?format=json' | jq -r '.ip' ) curl "http://my.api.com?query=$ipaddr" Note also that URLs should always ...


4

Best to use a json parsing tool to parse json. jq is a popular one. set -o pipefail # if supported by your shell var=$(curl -s "$url" | jq -r .result.progressId) || exit Or: json=$(curl -s "$url") || exit var=$(printf '%s\n' "$json" | jq -r .result.progressId) || exit (which allows you to extract more information from that json data later on).


3

Using jq, you would get the raw password string with curl ... | jq -r '.password[0][0]' I.e., get the first element of the array-in-array of the password key in the returned JSON object. Using sed in this instance would be a mistake, as the password string could possibly contain any character, including JSON encoded data that you would have to decode ...


3

Using a while read loop: while read -r ip; do curl --fail "ipinfo.io/${ip}/city" || break; done<ip_list.txt This will read through the ip_list.txt file and set each line to the ip loop variable, then curl it. Based on Stéphane's suggestion I have added the --fail flag to curl so it will fail silently upon a server error and the || break so that it will ...


3

You could use xargs and start a small shell script: jq -r '.href, .filename' file.json | xargs -n2 sh -c 'wget "$1" -O "$2"' sh Note that wget's -o option is for the logfile and -O is for the "output document".


3

Assuming you are using the default openssl implementation, you can query it directly: $ openssl version OpenSSL 1.1.1 11 Sep 2018 or more verbosely $ openssl version -a OpenSSL 1.1.1 11 Sep 2018 built on: Tue Nov 12 16:58:35 2019 UTC platform: debian-amd64 options: bn(64,64) rc4(16x,int) des(int) blowfish(ptr) compiler: gcc -fPIC -pthread -m64 -Wa,--...


3

There isn't a single answer: Firefox uses NSS to provide SSL/TLS A number of other programs use GnuTLS Still others use OpenSSL. And none of this tells you what version of SSL or TLS is actually in use, it just provides limits on what's available. For any given connection, the client and server negotiate a specific version of SSL/TLS and specific ciphers, ...


3

So ultimately it's because Github wants credentials. Without 2-factor auth, you can just do this with curl: curl -u username:password https://github.com/<org>/<repo>/tarball/<sha> but if you have 2-factor auth setup, then you need to use a Github access token, and you should use api.github.com instead of github.com, like so: curl -L "...


2

'curl' can make life easier. No root require ; curl is readily available on all the linux systems 1) If port is not open will show below output [niti@SourceServerName ~]$ curl -vv telnet://DestinationServerName:80 * About to connect() to DestinationServerName port 80 (#0) * Trying 192.168.0.100... 2) If port is open will show below output [niti@...


2

Due to https://mywiki.wooledge.org/BashPitfalls#Non-atomic_writes_with_xargs_-P (output from parallel jobs in xargs risks being mixed), I would use GNU Parallel instead: cat infile | parallel -P0 -q curl {} -o /dev/null --silent --head --write-out "%{http_code} {}\n" > outfile In this particular case it may be safe to use xargs because the output is ...


2

This will do what you want: wget -qO- http://website.com/site/ | \ sed -n -e "/^ *file: */ { s/^ *file: *'//; s/', *$//p}" > output.txt


2

If the actual URL contains an unquoted & character, the command up until that point will be interpreted by the shell as a command to run in the background. The rest of the URL would then be taken as a new command. Always quote URLs when specifying them on the command line. This also prevents other characters in the URL from being interpreted as special ...


2

For geographic IP address(es) lookup, I usually prefer using MaxMind's geoIP database. It is faster, as it does not an Internet connection for each IP address, as it uses a local database, and does not have limits on the number of calls. If in Debian, as a pre-requisite, install the packages: sudo apt-get install geoip-bin geoip-database-extra Then run ...


2

You forgot to close the double quoted string that you opened on the first line of your command (you used a single quote at the end of the URL, probably by mistake). Just press Ctrl+C and start over. Then make sure that you close each quote properly. The > prompt is the secondary prompt ($PS2). It is displayed in situation where more input is needed to ...


2

sudo pip install pip==8.1.2 It will install the 8.1.2 and remove the newest.


2

The Advanced Bash scripting guide states in chapter 5 that inside single quotes, the special meaning of $ is turned off, hence no variable expansion is performed. You could try enclosing your curl argument in double quotes " ... " instead of the single quotes.


Only top voted, non community-wiki answers of a minimum length are eligible