Hot answers tagged

3

So ultimately it's because Github wants credentials. Without 2-factor auth, you can just do this with curl: curl -u username:password https://github.com/<org>/<repo>/tarball/<sha> but if you have 2-factor auth setup, then you need to use a Github access token, and you should use api.github.com instead of github.com, like so: curl -L "...


2

Another approach to this is to use a GitHub cookie. It still starts with the normal user/pass, but after the initial request, you can make further requests by utilizing the cookie. Here is example with PHP: <?php $s_user = $argv[1]; $s_pass = $argv[2]; $r_url = curl_init(); curl_setopt($r_url, CURLOPT_COOKIEFILE, 'github.txt'); curl_setopt($r_url, ...


1

The CPU overhead of running a single GNU Parallel job is in the order of 1-10 ms. This is partly due to being written in Perl, but mostly due to a lot of safety tests happening behind the scenes. So if you want to use GNU Parallel to run 25000 jobs per second on average, you can do: seq 1000000 | parallel -n100 --pipe --round-robin -I ,, parallel myjob {} ...


1

Use the -R option of less, so that it interprets these codes as colour. It is rendering the codes in a human readable way. However can pass them to the terminal to be interpreted as colour. -r passes everything to terminal, -R only does this for colour. curl cheat.sh/python | less -R


1

Consider using %{url_effective} in the -w output fields: awk -v OFS== -v out=/dev/null \ '!/^(#|[[:blank:]]*$)/{print "url", $0; print "out", out}' urls.txt | curl -K- -sSLI -w '%{num_redirects},%{url_effective},%{http_code}\n' >> output.csv Because -L is used to follow redirects, there may be some discrepencies between the urls from urls.txt ...


1

By using echo and command substitution formatting of each line can be done like that: while IFS= read -r line do echo "$line,$(curl -LI "$line" -o /dev/null -w '%{http_code}\n' -s)" >> output.csv done < urls.txt


1

If you try to retrieve this file using a regular browser with Developer console launched, you'll notice that, when you click on the "link", it will trigger a HTTP POST request to https://www.fuelwatch.wa.gov.au/fuelwatch/pages/public/historicalFileDownloadRetail.jspx with the following parameters: { "j_idt72":"j_idt72", "j_idt72:resultsTbl_rppDD":"20",...


1

#!/bin/bash URL="https://www.gitignore.io/api/nonexistentlanguage" response=$(curl -s -w "%{http_code}" $URL) http_code=$(tail -n1 <<< "$response") # get the last line content=$(sed '$ d' <<< "$response") # get all but the last line which contains the status code echo "$http_code" echo "$content"


1

Part of your question was answered. In answer to the other part, the "two dashes" is typically the long form use of a parameter. It is common for commands to have both a long and a short form for any given parameter. For example, if you look at the manual for curl, you will see: -a, --append (FTP SFTP) When used in an upload, this makes curl... ...


Only top voted, non community-wiki answers of a minimum length are eligible