1

tl;dr

The command git fetch origin [branch-name] is not working when invoked within a script, but works when invoked from shell.

Both script and terminal are working just fine on another box. The box is behind a corporate proxy.

I assume there is some difference in the environment in the script compared to the shell.

Question

How can I get this command to run from within the script?

Detailed Info

Back Story

Everything worked on both boxes until the repository was migrated to GitHub. The origin was updated accordingly on both systems. The development system worked immediately, while the staging system just won't work.

Script

This is how the part of the script looks it checks all repositories are reached before acutally starting and therefor stops here:

check_git_connection_mapbender() {
  cd ${installation_folder}mapbender
  git fetch origin ${git_mapbender_repositoryBranch} &>/dev/null
  if [ $? -ne 0 ]; then
    echo "Fetching Mapbender repository failed!"
    echo "Abording update..."
    exit 1
  fi
  set_git_filemode_config
}

The two echo lines are echoed out and the script exits. The variables are all set.

What I've tried

compare shells

As I am suspecting some problem with environment variables I was trying to invoke the failing command git fetch origin [branch-name] from the terminal directly. The command worked in all the cases, when invoked from shell:

  1. me@box:/path/to/repo$ git fetch origin [branch-name]
  2. me@box:/path/to/repo$ sudo git fetch origin [branch-name]
  3. me@box:$ sudo -i cd /path/to/repo && git fetch origin [branch-name]
  4. me@box:$ sudo su - -> root@box: cd /path/to/repo && git fetch origin [branch-name]

What does not seem to work is the following way:

  1. me@box:$ sudo -i-> root@box: cd /path/to/repo && git fetch origin [branch-name] <-- this does throw the error like in the script

git config

At first I assured the effective git config is the same on both systems.

Since then I tried adding certain options on the staging system to fix the problem, but without success.

I tried setting:

  1. url.http://.insteadof=https://
  2. http.https://github.com.sslverify=false
  3. http.proxy
  4. https.proxy
  5. http.sslCert

As it didn't help I removed it again.

debugging connection

I added export GIT_TRACE_CURL=true to the script so it looks like this:

check_git_connection_mapbender() {
  cd ${installation_folder}mapbender
# DEBUG
  export GIT_TRACE_CURL=true
  source /etc/environment
  echo $http_proxy
  echo $https_proxy
# END DEBUG
  echo $git_mapbender_repositoryBranch
  git fetch origin ${git_mapbender_repositoryBranch} # &>/dev/null
  if [ $? -ne 0 ]; then
    echo "Fetching Mapbender repository failed!"
    echo "Abording update..."
    exit 1
  fi
  set_git_filemode_config
}

... and the output is

[correct http_proxy environment variable]
[correct https_proxy environment variable]
[correct branch name]
16:52:17.600248 http.c:599              == Info: Couldn't find host github.com in the .netrc file; using defaults
16:52:17.603973 http.c:599              == Info:   Trying 140.82.121.4...
16:52:17.604035 http.c:599              == Info: TCP_NODELAY set
16:52:17.605297 http.c:599              == Info: connect to 140.82.121.4 port 443 failed: Verbindungsaufbau abgelehnt
16:52:17.605372 http.c:599              == Info: Failed to connect to github.com port 443: Verbindungsaufbau abgelehnt
16:52:17.605386 http.c:599              == Info: Closing connection 0
fatal: unable to access 'https://github.com/LVGL-SL/mapbender-sl.git/': Failed to connect to github.com port 443: Verbindungsaufbau abgelehnt
Fetching Mapbender repository failed!
Abording update...

proxy configuration

I'm told that the systems should be subjected to the same firewall rule sets as there is one for the subnet.

14
  • 1
    How does the output from GIT_TRACE_CURL compare when it works, and when it doesn't? Do you have $http_proxy or $https_proxy set in your environment when it works, and unset when it doesn't? When you say it doesn't work in a script, is this script called from crontab, or does it fail when you run the script directly from your (otherwise working) command line?
    – roaima
    Aug 27, 2021 at 14:13
  • 1
    Does this answer your question? su vs sudo -s vs sudo -i vs sudo bash
    – alecxs
    Aug 27, 2021 at 14:45
  • 1
    @roaima I tried to improve the section
    – SvenTUM
    Aug 27, 2021 at 15:11
  • 1
    SvenTUM you can compare the environment (specifically for the proxy configuration) with env | grep -i proxy and then sudo -i / env | grep -i proxy. I suspect you've got settings in your usual account that are stripped out by sudo -i. Failing that, you've got git configuration under your normal $HOME but when you sudo -i the HOME variable is changed to that of the target user
    – roaima
    Aug 27, 2021 at 15:42
  • 2
    Wonderful! Please add that as an answer, which you can accept (possibly tomorrow), so that future readers can potentially benefit from your solution
    – roaima
    Aug 27, 2021 at 16:11

1 Answer 1

1

The box is behind a corporate proxy.

Couldn't find host github.com in the .netrc file; using defaults

connect to 140.82.121.4 port 443 failed: Verbindungsaufbau abgelehnt

The script is run with sudo and fails, but the command git fetch origin [branch-name] will work in the terminal

Guess: Your corporate proxy needs credentials which are stored somewhere under your user account, possibly in the .netrc file.

If you run the git command as normal user, it will access those credentials.

If you run the script as root, it will look for the credentials in root's home directory, cannot find them, and therefore fails.

You can verify by running the git command with sudo from the terminal; if the above is the reason, it will fail as well.

To fix your problem, you need to explain why you want to run a script as root that needs your normal user's credential to access github (which sounds like a fundamental design problem that should be addressed first). Depending on the explanation, you could give root credentials for proxy (unwise), impersonate a particular user inside the script to execute the git commands, or, and this is the best variant, fix whatever needs root access so it doesn't need root access, and can be run as normal user.

2
  • Hi @dirkt! Thank you for your answer. Currently the script does need root privileges for some other stuff, that's not related to git. There had been tons of improvements since I manage it, but it's still a journey to get it decent. There are no credentials for github needed, it's open source; the credentials for the proxy should be in the environment.
    – SvenTUM
    Aug 27, 2021 at 15:16
  • 2
    And the root environment (which you get via sudo) is different from the user's environment. So that's the first thing to investigate. That said, a script running under root needing to check out code from a git repository sounds just wrong, from a security perspective (you are in a corporate network, after all, and not at home). If you don't want to make the information why it needs root public, then talk to one of your sysadmins, and properly factor this for whatever it needs to do.
    – dirkt
    Aug 29, 2021 at 5:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.