4

I have this:

curl -L "https://github.com/cmtr/cp-go-api/tarball/$commit_id" | tar x -C "$project_dir/"

I am just trying to download a tarball from github and extract it to an existing directory. The problem is I am getting this error:

Step 10/13 : RUN curl -L "https://github.com/channelmeter/cp-go-api/tarball/$commit_id" | tar x -C "$project_dir/"
 ---> Running in a883449de956
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     9  100     9    0     0     35      0 --:--:-- --:--:-- --:--:--    35
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
The command '/bin/sh -c curl -L "https://github.com/channelmeter/cp-go-api/tarball/$commit_id" | tar x -C "$project_dir/"' returned a non-zero code: 2

does anyone know why it's not a tar archive? If you go to github.com in the browser and put in this pattern, it will download a tar.gz archive:

https://github.com/<org>/<repo>/tarball/<sha>

so not sure why it's not working.

4
  • 1
    What do you get if you got to this page (outside of a pipeline)? I can't find the project. Is it private? Mar 17, 2020 at 20:15
  • github.com/cmtr/cp-go-api/tarball doesn't work for me.
    – Time4Tea
    Mar 17, 2020 at 20:16
  • the repo is private, the actual org name is not cmtr Mar 17, 2020 at 20:18
  • 3
    it's possible that curl can't download it without credentials, so that's probably what is wrong Mar 17, 2020 at 20:18

2 Answers 2

6

So ultimately it's because Github wants credentials. Without 2-factor auth, you can just do this with curl:

curl -u username:password https://github.com/<org>/<repo>/tarball/<sha>

but if you have 2-factor auth setup, then you need to use a Github access token, and you should use api.github.com instead of github.com, like so:

 curl -L "https://api.github.com/repos/<org>/<repo>/tarball/$commit_sha?access_token=$github_token" | tar -xz -C "$extract_dir/"

the access token thing is documented here: https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line

1
  • 1
    query param access_token=$github_token is deprecated, looks like the suggestion is to use a header like curl -H "Authorization: token $github_token" ...
    – jmunsch
    Aug 19, 2020 at 17:13
3

Another approach to this is to use a GitHub cookie. It still starts with the normal user/pass, but after the initial request, you can make further requests by utilizing the cookie. Here is example with PHP:

<?php

# GET
$get = curl_init('https://github.com/login');
curl_setopt($get, CURLOPT_COOKIEJAR, 'github.txt');
curl_setopt($get, CURLOPT_RETURNTRANSFER, true);
$log = curl_exec($get);
curl_close($get);

# POST
preg_match('/name="authenticity_token" value="([^"]*)"/', $log, $auth);
$pf['authenticity_token'] = $auth[1];
$pf['login'] = getenv('USER');
$pf['password'] = getenv('PASS');
$post = curl_init('https://github.com/session');
curl_setopt($post, CURLOPT_COOKIEFILE, 'github.txt');
curl_setopt($post, CURLOPT_POSTFIELDS, $pf);
curl_exec($post);

Then after that, you can use shell cURL with -b github.txt, or PHP cURL with CURLOPT_COOKIEFILE github.txt. Make sure to curl_close as shown above, else the cookie file gets created after its needed.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .