Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Many times I have seen the note telling users not to update their repositories too often (i.e more than once a day) because that will put a stress too large for the servers. Also I understand that it takes monstrous machines for such repositories.

I am wondering if there is something like BitTorrent for package management? Or if there isn't, is it feasible to have such a system?

(I'm thinking about a system where each user keeps the packages that they have and serve it in the same fashion as BitTorrent.)

share|improve this question
    
I've never seen such a note. Also, that's the reason there are mirrors. –  Tshepang Jan 27 '11 at 12:29
    
@Tshepang Here is the one for Gentoo (at the bottom of the page), I also remember seeing it for another distro as well gentoo.org/doc/en/handbook/handbook-amd64.xml?part=2&chap=1 –  phunehehe Jan 27 '11 at 14:22
    
Perhaps that's unique to src-based systems? –  Tshepang Jan 27 '11 at 16:42
    
man, the correct answer should be to use appropriate cache headers, only fetch if newer... and I'm pretty sure that Arch has it somewhere... it's more for rolling distro's and people like me addicted to updates –  xenoterracide Jan 28 '11 at 10:09
add comment

4 Answers 4

up vote 6 down vote accepted

Indeed it's possible and of course it exists, but all of the existing projects are still just experiments.

I found two interesting papers:

So yes, it's possible and it's indeed a really interesting subject.

share|improve this answer
    
Look, this is the first answer of the kind that I was expecting :) –  phunehehe Jan 27 '11 at 16:38
    
@phunehehe thanks, I fixed a bit the answer. –  tmow Jan 28 '11 at 8:24
1  
something worth noting is that this would make international controls on crypto almost impossible...maybe why it hasn't been done - the US Govt for example considers crypto to be a "weaponized" tech. it would kind of suck to get arrested as a weapons trafficker for hosting a linux package. –  hbdgaf Jan 28 '11 at 15:09
    
@ aking1012 This is a good point, but the control is feasible, it's enough to download all the packages and verify them one by one (the same with a normal distribution), the important thing is that the GNU/Linux distributor provide a list of packages with a cksum (sha*, md5, whatever), that is what happen, anyway, in normal package distributions. –  tmow Jan 28 '11 at 16:34
add comment

Also, if this is within a corporate setting, you can setup local mirrors and/or proxies to alleviate this problem. There is no need for a bittorrent that way. If you are using a Debian based distribution, you can already just use apt-cacher/apt-proxy to do it.

share|improve this answer
    
Thanks, but this is not really about me or my company. I was just thinking of it as a way to avoid having stress on the server of any distribution. Would be fantastic, wouldn't it? –  phunehehe Jan 27 '11 at 14:40
add comment

You can try setting up a sort of apache based load balancer and specify several different mirrors as the "back" end. Each time you hit your proxy a different real mirror will be hit, thereby spreading your requests among several mirrors.

But, most mirrors only update once a day. A single mirroring job can be lengthy, especially when there's a large update. For mirror operators more than once per day usually is not worth it, and mirroring frequently puts much more stress on the upstream than a simple apt-get update.

Remember, mirrors and distributions are often done on a volunteer basis. Those volunteers have to pay big bucks for bandwidth, storage, electricity, cooling, etc.

I operate a Debian mirror myself. Please, help us out and don't slam us too hard.

share|improve this answer
add comment

For apt: apt-transport-debtorrent. Note that you still need to get the package list (i.e. apt-get update) from one of the mirrors so that it knows what packages are available .

share|improve this answer
    
I've tried using it and nothing happened. Has anyone used it successfully? –  Faheem Mitha Jun 6 '11 at 15:24
    
The last time I tried it worked, but was very slow. Try reading this: debtorrent.alioth.debian.org/Install –  syockit Jun 6 '11 at 22:57
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.