23

I have an Ubuntu 12.04 virtual box vm that I instantiate using Vagrant.

git clone https://github.com/spuder/puppet-gitlab
vagrant up

As soon as the vagrant box runs apt-get update, I get the following error.

...
W: Failed to fetch gzip:/var/lib/apt/lists/partial/apt.puppetlabs.com_dists_precise_main_binary-amd64_Packages  Hash Sum mismatch
W: Failed to fetch gzip:/var/lib/apt/lists/partial/apt.puppetlabs.com_dists_precise_main_binary-i386_Packages  Hash Sum mismatch
W: Failed to fetch gzip:/var/lib/apt/lists/partial/apt.puppetlabs.com_dists_precise_dependencies_binary-i386_Packages  Hash Sum mismatch
W: Failed to fetch http://br.archive.ubuntu.com/ubuntu/dists/precise-updates/restricted/binary-i386/Packages  404  Not Found

Things I've tried to work around this error.

  • Used 3 different ubuntu 12.04 boxes from 'http://www.vagrantbox.es'
  • solution suggested here:

    sudo rm -rf /var/lib/apt/lists/*
    sudo apt-get update
    sudo apt-get clean

  • Removed and readded the puppet labs packages

I've also tried similar suggestions that I've found in the first few pages of google. I've even tried multiple computers, and multiple internet connections.

The fact that this has affected multiple ubuntu vm's on multiple internet connections makes me think there is something wrong with the ubuntu repo.

How else can I try to fix this issue?

Update

I tried cleaning out '/var/lib/apt/lists/partial' and running apt-get clean then replaced the sources in /etc/sources/list by using the amazon mirrors suggested here:

I still get a similar error

Fetched 18.9 MB in 10s (1,865 kB/s)                                                                                                                                                                            
W: Failed to fetch bzip2:/var/lib/apt/lists/partial/us-west-1.ec2.archive.ubuntu.com_ubuntu_dists_precise_main_binary-amd64_Packages  Hash Sum mismatch
W: Failed to fetch bzip2:/var/lib/apt/lists/partial/us-west-1.ec2.archive.ubuntu.com_ubuntu_dists_precise_universe_binary-amd64_Packages  Hash Sum mismatch
W: Failed to fetch bzip2:/var/lib/apt/lists/partial/us-west-1.ec2.archive.ubuntu.com_ubuntu_dists_precise_multiverse_binary-amd64_Packages  Hash Sum mismatch
W: Failed to fetch bzip2:/var/lib/apt/lists/partial/us-west-1.ec2.archive.ubuntu.com_ubuntu_dists_precise_main_binary-i386_Packages  Hash Sum mismatch
E: Some index files failed to download. They have been ignored, or old ones used instead.

Update2

I have 2 internet connections at home, both of them gave the same error. As soon as I took my laptop to my work internet connection, the problem went away.

I'm still curious to understand why my internet connection would make any difference.

Update3

See my answer below for an explanation. My internet filter was corrupting the download.

I'll rephrase the question since there are no answers yet.

Is there a way to override Hash Sum mismatches in apt-get?

  • have you clean the cache as suggested here ? – Kiwy Feb 24 '14 at 10:52
  • Also does your distro include apt-cacher NG ? because it seems unreliable – Kiwy Feb 24 '14 at 11:01
  • I did come across that blog, and I verified that apt-cache-ng is not installed. The problem went away when I tried a different hypervisor on a different internet connection. – spuder Feb 24 '14 at 16:27
  • maybe a broken implementation of one specific hash on a hypervisor, but it's still very strange. – Kiwy Feb 24 '14 at 18:05
  • Your solution also worked for me. I was running through a TinyProxy server hosted on AWS and had some ad filters setup. Once I got rid of the proxy in apt.conf, everything works fine. – user81554 Aug 21 '14 at 15:37
17

First, you should understand why hash sum mismatch errors occur. In general, there are 2 reasons:

Firstly, Some apt repositories use LZMA (.xz) compressed metadata. apt before version 1.0 fails to decompressed LZMA archives correctly (sometimes) giving the hash sum mismatch error.

There's two work arounds for this:

  1. Tell your apt client not to use XZ compressed metadata
  2. Upgrade apt on your system to a version newer than 1.0

Check out this blog post I wrote about this issue which explains both work arounds in greater detail.

Secondly, APT repositories are inherently racy. The actual APT metadata is buggy and the design of it makes it impossible for apt clients to download the repository metadata in a consistent way if the apt-get update happens while the repository is being updated.

There's two work arounds for this:

  1. Upgrade to a newer version of APT and ensure that the repository you create (or want to use) supports the Acquire-by-hash feature. This fixes the issue at its core, but not be possible in some cases if you don't control the repository.
  2. You can delete the cached metadata on your system and try again. To do this, first run apt-get clean followed by rm -rf /var/lib/apt/lists/*. Next, run apt-get update. This will re-download all the metadata. If the repository you are trying to connect to doesn't update itself while you are running apt-get update, you will be fine. Otherwise, you'll have to do this again.
  • This is the only solution that worked for me! Thanks for the blog post – Amir Uval May 31 '16 at 13:56
  • You commented that apt better be the latest version. Could you please tell me how to update it? – Amir Uval May 31 '16 at 14:04
  • How exactly do you use the Acquire-by-hash feature? I've seen blogs about it but they don't say which file exactly to put it in and they don't show an example. – LINUX G33NYUS May 3 '17 at 23:49
6

Figured it out.

My two computers are identical in every way except on my personal computer I installed an internet filter.

The k9 internet filter installed a kernel extension that apparently messes with the traffic.

As soon as I uninstalled the filter, the problem went away.

I'm still researching if there is any way to make apt-get update ignore Hash sum mismatches as a workaround.

  • This actually makes a lot of sense, because I recently started having this issue after installing k9; but I didn't make the connection at the time. A workaround would be nice, because I'd obviously prefer not to disable the filter (after all, it's there for a reason). – MuffinTheMan Apr 18 '14 at 14:08
  • Yeah but how could i do that (remove the filter), because i have same problem of you, but anyhow im still stucked in – user89740 Oct 30 '14 at 13:49
  • Download the latest version of k9 from their website. There will be a uninstaller inside the installation package. – spuder Oct 30 '14 at 17:08

protected by Community Oct 30 '14 at 14:04

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.