3

i would like to create a rpm repository for fedora packages on my local network. due to storage limitations i want the repository to be empty initially and download packages once they are accessed.

Background

i work a lot with local VMs. Every time i create a new VM and install fedora a lot of packages are downloaded from the internet. Most of the downloaded packages are the same. To speed up the process i would like the RPMs to be cached on a server located on the same network.

similar questions have been answered with a combination of createrepo & reposync. i Do not like the reposync part, because i do not want to clone the whole repository in up-front when i only need some of the packages.

Ideal Solution

i would like the server on my local network to act as a rpm repository for my fedora installations. it should pass-through the metadata from whatever is configured in /etc/yum.repod.d/*. The server should deliver the requested RPM if it is present in the local cache or download it if it is not and deliver it then.

less ambitious approach would be to configure a single rpm repository instead of https://mirrors.fedoraproject.org/... and just use a http proxy.

Update: 02 Nov. 2015

i already have a nginx running on the network, so i played around with a combination of proxy_pass and proxy_cache. It kinda works, but has imho more drawbacks then benefits:

  • a separate configuration for every repo configured in /etc/yum.repo.d/*.
  • can't use metadata from https://mirrors.fedoraproject.org/ because of alternate mirrors.

i dropped the nginx thing and installed squid like suggested in comments. squid works great for me. With the store_id_program configuration i am even able to use the alternate mirrors and still hit the cache, no matter where the RPM came from originally.

  • 2
    The only thought in my head while reading your Q was "use squid" -- and then you said it – Jeff Schaller Oct 31 '15 at 21:08
  • yes, squid is the answer. note that some .rpms are quite large (e.g. kernels can be 30-40MB) so you'll have to increase squid's maximum_object_size from default 4MB to, say, 64MB. You might also want a refresh_pattern .rpm ... rule to keep rpm files longer than the default. – cas Oct 31 '15 at 23:31
  • btw, you can also use a squid url-rewriter to match the directory pattern of distro rpm repos and rewrite them to use just the one nearby mirror. – cas Oct 31 '15 at 23:33
  • and this LWN article from 2009 is old but still worth reading lwn.net/Articles/318658 – cas Oct 31 '15 at 23:36
  • and, finally, according to sotechdesign.com.au/… you can use Debian's apt-cacher-ng to cache yum repositories as well as .deb repos....but you have to do it in a debian vm because it probably isn't ported to fedora/rhel/centos/etc – cas Oct 31 '15 at 23:39
3

Here you can find fine-tuned squid.conf for rpm caching:

https://github.com/spacewalkproject/spacewalk/blob/master/proxy/installer/squid.conf

You just should modify memory and port setting.

1

I'm not sure if you found a good solution for this but I ended up writing a quick little Go program. It acts as a caching proxy and keeps around RPMs, but proxies upstream to database files so it's always correct. It's configured for CentOS, Fedora EPEL, and Arch Linux for now.

You can see it at https://github.com/yobert/remirror

0

Yum wiki contain page with review of caching solutions

Some of it on topic:

  • Install "IntelligentMirror", and register with MirrorManager.

    Pros
        Zero configuration, on the client side.
        Post setup it mostly just works and everything should be immediate.
        Only downloads what is required by the users.
        Fully automated server side, once setup/working. 
    Cons
        Requires setting up a local Squid + Apache-httpd + IntelligentMirror to serve the data.
        If the server is down then metalinks/MM will route you to an external mirror.
        Only intelligently caches packages, not metadata. 
    
  • (beta) avahi-packages-server.py and yum avahi support (see: http://james.fedorapeople.org/yum/avahi)

    Pros
        Zero configuration client.
        Zero configuration server.
        Only downloads what is required by the users. 
    Cons
        Not upstream yet (still beta).
        A client that doesn't run the avahi server side doesn't share it's downloads (so all clients need to run the "server").
        Requires working avahi on the network. 
    
0

apt-cacher-ng works great. We are using it for some times and it works fine. And remember to only use http mirrors (and not https ones) when using it to make it more efficient (naturally, it can't see & cache https traffic):

To configure fedora official repositories to only use http mirrors, you can add &protocol=http at the end of the metalink config url, e.g. for fedora repository it'll become:

metalink=https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=$basearch&protocol=http

https://www.unix-ag.uni-kl.de/~bloch/acng/

A caching proxy. Specialized for package files from Linux distributors, primarily for Debian (and Debian based) distributions but not limited to those. See documentation of Apt-Cacher to learn what it's good for.

  • Not sure why I got negative vote, but I've tried and it works fine. Probably the voter assumed that it only works for .deb, but if (s)he've read the link it'd be clear that it works for rpm repos too. – hedayat Dec 7 '18 at 8:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.