2

Yesterday I was setting up a new Kali guest VM in VirtualBox and had some issues during the installation. At the step of installing software packages the installation would fail, so after a few retries I decided to skip that step. The rest of the installation finnished without any issues.

After the installtion finnished I started trying to figure out which packages were missing. The first hitch I ran into was gettin apt to work. apt update and install would always fail so I cleand out /var/lib/apt and tried switching repo mirrors, but nothing helped. The specific error I get when running apt update is:

enter image description here

Then I noticed that the SHA checksums don't match but the MD5Sum actually does match. So my working hypothisis is that nothing is wrong with the downloads or repos, my system is producing wrong checksums and that's why apt always fails.

At this point I should probably nuke the VM and re-install the system, but I'd rather use this as a learning experience and fix the problem. So I'm hoping for sugestions on what to do next.

Edit in response to @Gilles 'SO- stop being evil' great answer.

I tried to verify if the Packages.gz file is out of sync with the metadata in InRelease.

root@kali:/var/lib/apt/lists/partial# rm *
root@kali:/var/lib/apt/lists/partial# apt update
Get:1 http://ftp.acc.umu.se/mirror/kali.org/kali kali-rolling InRelease [30.5 kB]
Get:2 http://ftp.acc.umu.se/mirror/kali.org/kali kali-rolling/main amd64 Packages [16.3 MB]                                    
Err:2 http://ftp.acc.umu.se/mirror/kali.org/kali kali-rolling/main amd64 Packages
  Hash Sum mismatch
  Hashes of expected file:
   - Filesize:16317378 [weak]
   - SHA256:77a3e22e7b5ea34fca2d74d79a9d46f4bb27af0dfb56d6052e2d288b3c684d98
   - SHA1:f5b21d796c25dc10d382ffedc1ce4d7bee376057 [weak]
   - MD5Sum:257a18dc4dff52c27f94f6e66a5a82bf [weak]
  Hashes of received file:
   - SHA256:5d1d8ffe97ff7a35ce5537925d7790967b086c75dadd5576688c915830bf0c84
   - SHA1:ce0617edf0193841072c1cba00b6797d2b3dd0eb [weak]
   - MD5Sum:257a18dc4dff52c27f94f6e66a5a82bf [weak]
   - Filesize:16317378 [weak]
  Last modification reported: Fri, 03 Apr 2020 15:48:14 +0000
  Release file created at: Fri, 03 Apr 2020 15:48:24 +0000

Fetched 16.3 MB in 5s (3368 kB/s)
Failed to fetch http://ftp.acc.umu.se/mirror/kali.org/kali/dists/kali-rolling/main/binary-amd64/Packages.gz
  Hash Sum mismatch
   Hashes of expected file:
    - Filesize:16317378 [weak]
    - SHA256:77a3e22e7b5ea34fca2d74d79a9d46f4bb27af0dfb56d6052e2d288b3c684d98
    - SHA1:f5b21d796c25dc10d382ffedc1ce4d7bee376057 [weak]
    - MD5Sum:257a18dc4dff52c27f94f6e66a5a82bf [weak]
   Hashes of received file:
    - SHA256:5d1d8ffe97ff7a35ce5537925d7790967b086c75dadd5576688c915830bf0c84
    - SHA1:ce0617edf0193841072c1cba00b6797d2b3dd0eb [weak]
    - MD5Sum:257a18dc4dff52c27f94f6e66a5a82bf [weak]
    - Filesize:16317378 [weak]
   Last modification reported: Fri, 03 Apr 2020 15:48:14 +0000
   Release file created at: Fri, 03 Apr 2020 15:48:24 +0000[0m
    Some index files failed to download. They have been ignored, or old ones used instead.[0m
root@kali:/var/lib/apt/lists/partial# ls
ftp.acc.umu.se_mirror_kali.org_kali_dists_kali-rolling_InRelease
ftp.acc.umu.se_mirror_kali.org_kali_dists_kali-rolling_main_binary-amd64_Packages.gz.FAILED

root@kali:/var/lib/apt/lists/partial# md5sum ftp.acc.umu.se_mirror_kali.org_kali_dists_kali-rolling_ main_binary-amd64_Packages.gz.FAILED 
257a18dc4dff52c27f94f6e66a5a82bf  ftp.acc.umu.se_mirror_kali.org_kali_dists_kali-rolling_main_binary-amd64_Packages.gz.FAILED

root@kali:/var/lib/apt/lists/partial# sha1sum  ftp.acc.umu.se_mirror_kali.org_kali_dists_kali-rollingg_main_binary-amd64_Packages.gz.FAILED 
f5b21d796c25dc10d382ffedc1ce4d7bee376057  ftp.acc.umu.se_mirror_kali.org_kali_dists_kali-rolling_main_binary-amd64_Packages.gz.FAILED

root@kali:/var/lib/apt/lists/partial# sha256sum ftp.acc.umu.se_mirror_kali.org_kali_dists_kali-rollinng_main_binary-amd64_Packages.gz.FAILED 
77a3e22e7b5ea34fca2d74d79a9d46f4bb27af0dfb56d6052e2d288b3c684d98  ftp.acc.umu.se_mirror_kali.org_kali_dists_kali-rolling_main_binary-amd64_Packages.gz.FAILED

So as far as I can tell, the Packages.gz file is downloaded correctly and the actuall hashes do match what is expected from the InRelease file. But apt still reports wrong hashes.

Edit 2:

So, after mucking about a lot I did finally get apt to a working state by manually downgrading apt to version 1.8.4 (original version was 2.0.2). The issue is reproducible, running apt upgrade installs 2.0.2 and the problem returns.

  • 1
    @GAD3R No, as I said I already tried switching repo mirrors but they all gave the same error, correct md5 but mismatching sha256 and sha1. – JonC Apr 4 at 13:51
  • I am having the same issues described here. How did you go about downgrading to 1.8.4? – L337BEAN Apr 25 at 16:29
  • 1
    @L337BEAN This is a bit dangerous so don't try it without a proper backup of your system. I manually downloaded the deb package for apt from the Kali repo, uninstalled apt with "apt remove apt", then installed the downloaded version using "dpkg -i apt_1.8.4_amd64.deb". – JonC Apr 26 at 11:28
  • Thanks for the info. I went ahead a built a new VM under Hyper-V and I'm having no issues with the repos. – L337BEAN May 1 at 15:06
  • 1
    I had the same issue. Check out this answer here, he solves it. Particularly for Windows 10 host using Virtualbox, so I'm wondering if that's the issue here. For me when I asked that question, some packages worked fine but others did not. The key is disabling Hyper-V and getting rid of that Green Turtle – RyanQuey May 23 at 0:01
1

This answer assumes a Windows 10 host.

I've encountered what appears to be the same "Hash Sum mismatch" error on "Packages.gz" during the "Installing the base system" step of any of the 2020-2 amd-64 ISOs on VirtualBox. I also booted the Kali 2020-2 amd-64 VirtualBox OVA, and received the same error while attempting an apt-get update. It appears to have been resolved for me by disabling the "Windows Defender Credential Guard" feature, otherwise known as "Device Guard" or "Virtualization Based Security".

Manage Windows Defender Credential Guard

Introduced in Windows 10 Enterprise and Windows Server 2016, Windows Defender Credential Guard uses virtualization-based security to isolate secrets so that only privileged system software can access them. Unauthorized access to these secrets can lead to credential theft attacks, such as Pass-the-Hash or Pass-The-Ticket. Windows Defender Credential Guard prevents these attacks by protecting NTLM password hashes, Kerberos Ticket Granting Tickets, and credentials stored by applications as domain credentials. Reference Link

There are several methods of disabling this feature, as explained in the link. I used the "Windows Defender Credential Guard hardware readiness tool", available here.

DG_Readiness_Tool_v3.6.ps1 -Disable -AutoReboot
| improve this answer | |
  • I didn't have credential guard, but I still have the issue no matter the repo I use. There must be something wrong with the Kali package archive. – mbomb007 9 hours ago
3

The Kali package archive is currently in an inconsistent state. You can't do anything about it.

It would be very unlikely that your system would produce wrong checksums. There are several reasons it could happen, but none of them are plausible.

  • The software that calculates the checksum themselves could be buggy. This is extremely unlikely: calculating checksums is easy and the code to do so is very stable and easy to test.
  • The software that downloads the files, stores them, verifies them and so on could be buggy. It's very unlikely that it would be buggy in such a way that it would calculate wrong checksums, as opposed to erroring out.
  • The software might be downloading the wrong files, or truncating them or encoding them in a way that isn't caught. This is the least implausible bullet point here.
  • Your system could be compromised in such a way as to calculate wrong checksums. This is implausible because an attacker who can do that can do far more useful things in a less conspicuous way.

It's somewhat less unlikely that your network is under attack, and an attacker is actively manipulating the files that you're downloading. It's still unlikely because the attacker would know that the attack would be detected and ineffective due to the cryptographic checks that apt makes (I'll explain these checks below). The attack would only be useful against a user who arranges to ignore the error or who manually downloads .deb files and installs them with dpkg.

Unlikely doesn't mean impossible, of course. You can verify that none of this is happening by downloading the files and calculating their checksum on a different, known good system. I did that and got the same values of expected and actual checksums.

The corruption could be in one mirror, so I used a different mirror (https://http.kali.org/dists/kali-rolling/). The InRelease file contains the expected checksums, and Packages.gz is the file whose checksums are verified.

$ wget -q https://http.kali.org/dists/kali-rolling/InRelease https://http.kali.org/dists/kali-rolling/main/binary-arm64/Packages.gz
$ TZ=UTC \ls -log InRelease Packages.gz
-rw-rw-r-- 1    30501 Apr  3 15:48 InRelease
-rw-rw-r-- 1    30501 Apr  3 15:48 InRelease
-rw-rw-r-- 1 16179052 Apr  3 12:04 Packages.gz
$ md5sum Packages.gz
31a332531ecf9d092aaad9a3f4885767  Packages.gz
$ sha1sum Packages.gz
138883655ff0d58a3779acbeda0d61f7552c03eb  Packages.gz
$ sha256sum Packages.gz
63ae17c54bc57dc445ba4a3555bec3fa077c5de6eec0b11363680efc23fd09ec  Packages.gz
$ grep main/binary-amd64/Packages.gz InRelease
 257a18dc4dff52c27f94f6e66a5a82bf 16317378 main/binary-amd64/Packages.gz
 f5b21d796c25dc10d382ffedc1ce4d7bee376057 16317378 main/binary-amd64/Packages.g
 77a3e22e7b5ea34fca2d74d79a9d46f4bb27af0dfb56d6052e2d288b3c684d98 16317378 main/binary-amd64/Packages.gz

As you can see, the expected and actual checksums are different. The expected and actual sizes are also different. I have a different, older version of Packages.gz than you, even though I downloaded more recently, but from a different mirror.

I also downloaded the files from the same mirror as you and there the files had their expected checksums, so the problem is repaired on that mirror. It looks like a temporary error and the fix hasn't been fully propagated yet.

I don't know what caused the problem. It could be an attack attempt (but if so it seems to have failed since not all the files that needed to be corrupted were corrupted). More likely, it was a synchronization failure somewhere inside the Kali infrastructure.

I don't know why you're seeing a matching MD5. Either the InRelease file that you downloaded has inconsistent data or apt is not even bothering to calculate MD5 because it's considered weak.

As promised, here's how apt ensures the security of downloads. The following cryptographic infrastructure generates the data that guarantees that the packages are genuine:

  • A build server calculates the cryptographic hash¹ of each package (.deb, or file of a source package).
  • A hashing server builds the package list (Packages, and a compressed version Packages.gz) from the hashes sent by the build server for each part of the distribution, and generates a Release file containing the hashes of the Packages files.
  • A signing server, which has a PGP private key, generates a cryptographic signature of the Release file and stores it in Release.gpg. There's also a file InRelease which contains both the data and the signature in the same file.

On your system:

  • Your initial installation image contains the PGP public key for the build server's private key, as well as all the tools necessary to validate that a file is correctly signed with this key.
  • When apt downloads the package list, it downloads the InRelease file (or maybe Release and Release.gpg) and verifies that it is correctly signed. It also verifies that the cryptographic hashes of the Package files match the value in the InRelease file.
  • When apt downloads a package, it verifies that the hashes of the package file match the values in the Packages file.

This is sufficient because:

  • No one knows how to make a file with the same cryptographic hash as another existing file. (This is true even for MD5 and SHA-1, for which we know how to make a collision, i.e. how to make two files with the same hash, but not how to calculate a second preimage, i.e. to find another file whose hash is the same as a given file.)
  • No one knows how to generate a valid PGP signature without having the private key.

That's it. Note that it doesn't matter how the files were transferred between the Kali infrastructure and the download mirror, or between the download mirror and your system. Using TLS for those is a security improvement because it prevents a network attacker from serving stale files (for example, from pretending that a security update to a critical piece of software never happened, by serving genuine but obsolete packages with the corresponding obsolete version of the Release file and its signature).

The only way something can go undetected is inside the Kali infrastructure: if the signing key is compromised, or if the build servers report the wrong hashes.

¹ In this context, “(cryptographic) checksum”, “(cryptographic) hash” and “(cryptographic) digest” are synonyms. There are non-cryptographic checksums and hashes but they aren't involved here.

| improve this answer | |
  • This is a great and thurough answer and it really helped me to better understand how the repo is accessed and handled by apt. But I try to verify the actual hashes it still looks like apt calculates the SHA hashes wrong. I've updated my question with more info. – JonC Apr 5 at 18:06
  • But since the MD5-sums are correct, it can't really be corruption during transmission. It is exceedingly unlikely that the MD5-sum wouldn't become wrong too. It could be an attack, but I don't think anyone has managed SHA256 yet. It really has to be something with the calculation of the checksum. – avl_sweden May 28 at 6:31
0

Disabling Hyper-V

Disabling Hyper-V worked for me.

Per this comment, I found the resources I needed to do just that.

  1. Open an elevated cmd prompt
  2. Run bcdedit and check the setting for hypervisorlaunchtype under {current}
  3. Run bcdedit /set {current} hypervisorlaunchtype off
  4. Reboot

After doing this, I no longer see the "green turtle" on the guest's status bar.

To turn it back on, do the process save as above with this alternate step 3:

  1. Run bcdedit /set {current} hypervisorlaunchtype auto

https://www.tenforums.com/tutorials/139405-run-hyper-v-virtualbox-vmware-same-computer.html#Part1

Note:

I noticed that Docker For Windows uses Hyper-V, so it's possible that shutting down Docker would fix the VBox issues if you are using Docker.

| improve this answer | |
New contributor
mbomb007 is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.