10

Many times I have seen the note telling users not to update their repositories too often (i.e more than once a day) because that will put a stress too large for the servers. Also I understand that it takes monstrous machines for such repositories.

I am wondering if there is something like BitTorrent for package management? Or if there isn't, is it feasible to have such a system?

(I'm thinking about a system where each user keeps the packages that they have and serve it in the same fashion as BitTorrent.)

4
  • 1
    I've never seen such a note. Also, that's the reason there are mirrors.
    – tshepang
    Jan 27, 2011 at 12:29
  • @Tshepang Here is the one for Gentoo (at the bottom of the page), I also remember seeing it for another distro as well gentoo.org/doc/en/handbook/handbook-amd64.xml?part=2&chap=1
    – phunehehe
    Jan 27, 2011 at 14:22
  • Perhaps that's unique to src-based systems?
    – tshepang
    Jan 27, 2011 at 16:42
  • man, the correct answer should be to use appropriate cache headers, only fetch if newer... and I'm pretty sure that Arch has it somewhere... it's more for rolling distro's and people like me addicted to updates Jan 28, 2011 at 10:09

4 Answers 4

8

Indeed it's possible and of course it exists, but all of the existing projects are still just experiments.

I found two interesting papers:

So yes, it's possible and it's indeed a really interesting subject.

5
  • Look, this is the first answer of the kind that I was expecting :)
    – phunehehe
    Jan 27, 2011 at 16:38
  • @phunehehe thanks, I fixed a bit the answer.
    – tmow
    Jan 28, 2011 at 8:24
  • 1
    something worth noting is that this would make international controls on crypto almost impossible...maybe why it hasn't been done - the US Govt for example considers crypto to be a "weaponized" tech. it would kind of suck to get arrested as a weapons trafficker for hosting a linux package. Jan 28, 2011 at 15:09
  • @ aking1012 This is a good point, but the control is feasible, it's enough to download all the packages and verify them one by one (the same with a normal distribution), the important thing is that the GNU/Linux distributor provide a list of packages with a cksum (sha*, md5, whatever), that is what happen, anyway, in normal package distributions.
    – tmow
    Jan 28, 2011 at 16:34
  • I was able to find the first paper ( www2.cs.uh.edu/~paris/MYPAPERS/P2p08.pdf ), but does anybody know where to find the second one eleven years later?
    – optical
    Oct 3, 2022 at 3:11
4

Also, if this is within a corporate setting, you can setup local mirrors and/or proxies to alleviate this problem. There is no need for a bittorrent that way. If you are using a Debian based distribution, you can already just use apt-cacher/apt-proxy to do it.

1
  • Thanks, but this is not really about me or my company. I was just thinking of it as a way to avoid having stress on the server of any distribution. Would be fantastic, wouldn't it?
    – phunehehe
    Jan 27, 2011 at 14:40
1

You can try setting up a sort of apache based load balancer and specify several different mirrors as the "back" end. Each time you hit your proxy a different real mirror will be hit, thereby spreading your requests among several mirrors.

But, most mirrors only update once a day. A single mirroring job can be lengthy, especially when there's a large update. For mirror operators more than once per day usually is not worth it, and mirroring frequently puts much more stress on the upstream than a simple apt-get update.

Remember, mirrors and distributions are often done on a volunteer basis. Those volunteers have to pay big bucks for bandwidth, storage, electricity, cooling, etc.

I operate a Debian mirror myself. Please, help us out and don't slam us too hard.

0
1

For apt: apt-transport-debtorrent. Note that you still need to get the package list (i.e. apt-get update) from one of the mirrors so that it knows what packages are available .

2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .