What's the 'right' format for the HTTP_PROXY environment variable? Caps or no caps?

Question

I've encountered both http_proxy and HTTP_PROXY. Are both forms equivalent? Does one of them take precedence over the other?

Answer 1

There is no central authority who assigns an official meaning to environment variables before applications can use them. POSIX defines the meaning of some variables (PATH, TERM, …) and lists several more in a non-normative way as being in common use, all of them in uppercase. http_proxy and friends isn't one of them.

Unlike basically all conventional environment variables used by many applications, http_proxy, https_proxy, ftp_proxy and no_proxy are commonly lowercase. I don't recall any program that only understands them in uppercase, I can't even find one that tries them in uppercase. Many programs use the lowercase variant only, including lynx, wget, curl, perl LWP, perl WWW::Search, python urllib/urllib2, etc. So for these variables, the right form is the lowercase one.

The lowercase name dates back at least to CERN libwww 2.15 in March 1994 (thanks to Stéphane Chazelas for locating this). I don't know what motivated the choice of lowercase, which would have been unusual even then.

Answer 2

There is no standard and both uppercase and lowercase versions are used depending on the application (also see HTTPS_PROXY, ALL_PROXY, NO_PROXY).

For example:

curl

ENVIRONMENT VARIABLES

  Curl reads and understands the following environment variables:

        http_proxy, HTTPS_PROXY, FTP_PROXY

  They should be set for protocol-specific proxies. General proxy should be
  set with

        ALL_PROXY

  A comma-separated list of host names that shouldn't go through any proxy is
  set in (only an asterisk, '*' matches all hosts)

        NO_PROXY

git

http.proxy
   Override the HTTP proxy, normally configured using the http_proxy, https_proxy, 
   and all_proxy environment variables (see curl(1)). [..]

Python

urllib.request.getproxies() supports both lowercase and uppercase variants.

It also mentions a security issue:

If the environment variable REQUEST_METHOD is set, which usually indicates your script is running in a CGI environment, the environment variable HTTP_PROXY (uppercase _PROXY) will be ignored. This is because that variable can be injected by a client using the “Proxy:” HTTP header. If you need to use an HTTP proxy in a CGI environment, either use ProxyHandler explicitly, or make sure the variable name is in lowercase (or at least the _proxy suffix).

---

Some applications allow NO_PROXY to contain stars/ip-ranges while others do not.

---

So

export https_proxy=$http_proxy HTTP_PROXY=$http_proxy HTTPS_PROXY=$http_proxy NO_PROXY=$no_proxy

should have you covered.

Answer 3

Most applications seem to support upper case variables, and others support lower case. I think some of the confusion comes from the curl removing support for HTTP_PROXY as it could be set via an HTTP Proxy: header in a CGI-BIN or PHP-type environment. Source code from curl:

https://github.com/curl/curl/blob/30e7641d7d2eb46c0b67c0c495a0ea7e52333ee2/lib/url.c#L2250-L2266

GitLab Blog post describing this madness:

https://about.gitlab.com/blog/2021/01/27/we-need-to-talk-no-proxy/

Answer 4

In general, environment variables are written in uppercase. http_proxy is exception sometimes. For instance, variable HTTP_PROXY can be controlled by attackers in CGI environments, as in golang/go#16405:

• The CGI spec defines that incoming request header "Foo: Bar" maps to environment variable HTTP_FOO == "Bar". (see RFC 3875 4.1.18)

• The HTTP_PROXY environment variable is conventionally used to configure the HTTP proxy for HTTP clients (and is respected by default for Go's net/http.Client and Transport)

So it really depends on a program (of course they may uses proxy settings in rc or config file):

• curl uses http_proxy, HTTPS_PROXY, NO_PROXY (reference)
• wget uses http_proxy, https_proxy, no_proxy (reference)

You may set up environment variable in both cases to be sure, for example:

export http_proxy=http://proxy.my.com:8080
export https_proxy=http://proxy.my.com:8080
export no_proxy=.host1,.my.com,.local
# reuse values for another case
export HTTP_PROXY=${http_proxy}
export HTTPS_PROXY=${https_proxy}
export NO_PROXY=${no_proxy}

Answer 5

WGET - is only using the lowercase proxy settings!

Answer 6

You can use this script to manage shell proxy on the go...

set_proxy() {
    type="$1"
    host="$2"
    port="$3"
    user="$4"
    pass="$5"
    if [[ -n "$user" ]]; then
        export http_proxy="$type"://"$user":"$pass"@"$host":"$port"
    else
        export http_proxy="$type"://"$host":"$port"
    fi
    export HTTP_PROXY=$http_proxy
    
    export https_proxy=$http_proxy
    export HTTPS_PROXY=$http_proxy
    
    export ftp_proxy=$http_proxy
    export FTP_PROXY=$http_proxy
    
    export all_proxy=$http_proxy
    export ALL_PROXY=$http_proxy
    
    echo "$http_proxy"
}
    
unset_proxy() {
    unset http_proxy
    unset HTTP_PROXY
    
    unset https_proxy
    unset HTTPS_PROXY
    
    unset ftp_proxy
    unset FTP_PROXY
    
    unset all_proxy
    unset ALL_PROXY
}

for example, to set a proxy, run

set_proxy socks5 127.0.0.1 10802

to disable the proxy, run

unset_proxy

Answer 7

For resource saving reasons I also encountered this issue, with question if Docker builds can be speed up by providing a proxy environment variable.

After discovering, that providing HTTP_PROXY does not help at all, started digging a bit, and here is some details:

If environment variable http_proxy (lowercase) is set, its used by:

• curl
• wget
• apt-get (apt also)
• Lynx browser

To sum up, Docker file should look like:

FROM python:3.12-slim-bookworm
ARG HTTP_PROXY=
ARG http_proxy=${HTTP_PROXY}
ARG HTTPS_PROXY=

Also note HTTPS_PROXY= line, if you do not want HTTPS to go via proxy, as requests are not cached, you should explicitly set it to nothing. Note also, that I use caps case argument name in order to be uniform with other env variables.

Do not forget to build Docker image with argument provided, like this:

docker build ..... --build-arg "HTTP_PROXY=$HTTP_PROXY" .

Still some note regarding python and node. pip and npm use https as source repository protocol, thus other caching techniques needs to be used.

In regular linux .profile you can set proxy like this:

export HTTP_PROXY=http://myproxyserver
export http_proxy=${HTTP_PROXY}

Answer 8

Both http_proxy and HTTP_PROXY should work on RHEL and ubuntu based systems.

Answer 9

The convention is to use all capps environment variables when exporting them, so that when you are writing shell scripts you can use lowercase variable names without worrying about name collisions with other programs. Of course this is a convention only, there is no technical restriction on limiting the names of environment variables so the lowercase version could be used in some cases, but best practice is upper case, and remember that they are case sensitive so they could have different values.