I've got a simple encrypted file with a bit of text inside. It's encrypted with des3 and I know the key. However, I can't for the life of me get it to decrypt in my kali VM. It works fine in a LInux Mint VM however. I'm at my wits end here.... What am I doing wrong?

Here's the working decryption:

user@user-virtual-machine ~/Desktop $ openssl des3 -d -in TheKeyIsInHere.des3 -pass pass:aramisthethird
GJC13 says the key is nuorjbwyldurrurykpym
user@user-virtual-machine ~/Desktop $

And here's the broken one:

root@chkali:~/Desktop/new# openssl des3 -d -in TheKeyIsInHere.des3 -pass pass:aramisthethird
bad decrypt
139786246681728:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:crypto/evp/evp_enc.c:529:

The file is the same in both cases (verified by md5sum).

  • I once spent ages troubleshooting a similar issue where I mistyped a 0 as a o. I doubt it's the same issue here, but try if you can to make sure you've cut/pasted the command rather than retyping. – SauceCode Feb 11 '17 at 1:21
  • Also maybe openssl version on both machines is worth checking. – SauceCode Feb 11 '17 at 1:22
  • Also it looks like you've run one command as root, and the other not. Although that's probably not relevant, since the successful one was non-root, try them in the most comparable state you can. Finally (as you probably know) you're running them in different directories. – SauceCode Feb 11 '17 at 1:29
  • 4
    Is one of them 1.1.0 and the other not? The default hash for the KDF used by enc changed in 1.1.0; to get old behavior on a new version specify -md md5 and for new behavior on an old version -md sha256 (supported at least back to 0.9.8 but not documented then). – dave_thompson_085 Feb 11 '17 at 9:18
  • 1
    We have a winner! @dave_thompson_085, that's exactly what the problem was. I just added -md md5 to the command, and it worked perfectly. Thanks! If you want to write an answer, I'll accept it. If not, I'll answer it myself, just so there's an answer here. – Carrot Feb 11 '17 at 23:15

The default hash used by openssl enc for password-based key derivation changed in 1.1.0 to SHA256 versus MD5 in lower versions. This produces a different key from the same password (and salt if used as it usually is), and trying to encrypt and decrypt with different keys produces garbage, an error, or both.

To fix this for existing data specify -md md5 in 1.1.0 to decrypt data from lower versions, and -md sha256 in lower versions to decrypt data from 1.1.0. Going forward, consider specifying -md explicitly. For details see https://crypto.stackexchange.com/questions/3298/is-there-a-standard-for-openssl-interoperable-aes-encryption/35614#35614 (disclosure: mine)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.