getting debian packages from elastic.co through apt-cacher-ng

Question

I am trying to use apt-cacher-ng to fetch packages for elasticsearch, kibana and logstash and am having no luck. This cache server shall not function as a general proxy so the PassThroughPattern: .* does not sound like a good idea to me.

I have tried to do a rewrite rule in /etc/apt-cacher-ng/acng.conf (notice the rewrite from http to https)

Remap-elastic: http://artifacts.elastic.co ; https://artifacts.elastic.co

and on the client I had the following line in `/etc/apt/apt.conf.d/02proxy

Acquire::http::Proxy "http://192.168.140.18:3142";

and in /etc/apt/sources.list.d/elastic-7.x.list I had

deb http://artifacts.elastic.co/packages/7.x/apt stable main

When I do an apt update on the client results in

Err:4 http://artifacts.elastic.co/packages/7.x/apt stable InRelease
  Connection failed [IP: 192.168.140.18 3142]
Reading package lists... Done
Building dependency tree       
Reading state information... Done
All packages are up to date.
W: Failed to fetch http://artifacts.elastic.co/packages/7.x/apt/dists/stable/InRelease  Connection failed [IP: 192.168.140.18 3142]
W: Some index files failed to download. They have been ignored, or old ones used instead.

and on the server with Debug option set to 7 and Verbosity to 2 I get in /var/log/apt-cacher-ng/apt-cacher.err

Thu Jul  1 12:22:44 2021|Returning to last state, 6
Thu Jul  1 12:23:10 2021|fileitem::DoDelayedUnregAndCheck, nextRunTime now: 1625134998
Thu Jul  1 12:23:13 2021|Detected incoming connection from the TCP socket
Thu Jul  1 12:23:13 2021|Client name: 192.168.140.5
Thu Jul  1 12:23:13 2021|Decoded request URI: http://artifacts.elastic.co/packages/7.x/apt/dists/stable/InRelease
Thu Jul  1 12:23:13 2021|Processing new job, GET http://artifacts.elastic.co/packages/7.x/apt/dists/stable/InRelease HTTP/1.1
Thu Jul  1 12:23:15 2021|Outgoing connection for Connection timed outartifacts.elastic.co, Port: 443
Thu Jul  1 12:23:15 2021|Outgoing connection for Network is unreachableartifacts.elastic.co, Port: 443
Thu Jul  1 12:23:16 2021|tcpconnect::ExpireCache, nextRunTime now: 1625135005
Thu Jul  1 12:23:18 2021|fileitem::DoDelayedUnregAndCheck, nextRunTime now: 9223372036854775805
Thu Jul  1 12:23:25 2021|tcpconnect::ExpireCache, nextRunTime now: 9223372036854775805
Thu Jul  1 12:23:45 2021|Outgoing connection for Connection timed outartifacts.elastic.co, Port: 443
Thu Jul  1 12:23:45 2021|Outgoing connection for Network is unreachableartifacts.elastic.co, Port: 443

So I tried the method the author of apt-cacher-ng prefers, I do not have any specific rewrite for elastic.co on the server this time around but on the client in the file /etc/apt/sources.list.d/elastic-7.x.list I had

deb http://192.168.140.18:3142/HTTPS///artifacts.elastic.co/packages/7.x/apt stable main

Doing an apt update on client results in

Err:4 http://192.168.140.18:3142/HTTPS///artifacts.elastic.co/packages/7.x/apt stable InRelease
  Connection failed [IP: 192.168.140.18 3142]
Reading package lists... Done
Building dependency tree       
Reading state information... Done
All packages are up to date.
W: Failed to fetch http://192.168.140.18:3142/HTTPS///artifacts.elastic.co/packages/7.x/apt/dists/stable/InRelease  Connection failed [IP: 192.168.140.18 3142]
W: Some index files failed to download. They have been ignored, or old ones used instead.

and on server in /var/log/apt-cacher-ng/apt-cacher.err with Debug option set to 7 and Verbosity to 2 I get

Thu Jul  1 11:40:39 2021|Returning to last state, 6
Thu Jul  1 11:40:39 2021|Decoded request URI: /HTTPS///artifacts.elastic.co/packages/7.x/apt/dists/stable/InRelease
Thu Jul  1 11:40:39 2021|Processing new job, GET /HTTPS///artifacts.elastic.co/packages/7.x/apt/dists/stable/InRelease HTTP/1.1
Thu Jul  1 11:41:09 2021|Detected incoming connection from the TCP socket
Thu Jul  1 11:41:09 2021|Client name: 192.168.140.5
Thu Jul  1 11:41:09 2021|Decoded request URI: /HTTPS///artifacts.elastic.co/packages/7.x/apt/dists/stable/InRelease
Thu Jul  1 11:41:09 2021|Processing new job, GET /HTTPS///artifacts.elastic.co/packages/7.x/apt/dists/stable/InRelease HTTP/1.1
Thu Jul  1 11:41:10 2021|Outgoing connection for Connection timed outartifacts.elastic.co, Port: 443
Thu Jul  1 11:41:10 2021|Outgoing connection for Network is unreachableartifacts.elastic.co, Port: 443
Thu Jul  1 11:41:40 2021|Outgoing connection for Connection timed outartifacts.elastic.co, Port: 443
Thu Jul  1 11:41:40 2021|Outgoing connection for Network is unreachableartifacts.elastic.co, Port: 443

Except for the Debug and Verbosity all the other options in acng.conf file are at its default and it works fine for normal repos like debian's but I can't get it to work for elastic or proxmox.

All ideas welcome.

Answer 1

This was stupid of me.

I tried to download the InRelease file from the elastic.co server from the command line (something I should have begun with) and found out it was blocked.

Looked in firewall rules at Proxmox level and lo and behold, port 443 was not allowed (default rule is block). Allowing outgoing traffic on port 443 and then the first version of what I tried works fine.

If it isn't DNS, its firewall right?