I have to encrypt a hash, eg:


using the RSA algorithm by using a public key(eg: mykey.key) that was already provided to me. How do I go about doing this. I see sites telling me to use openssl, but I'm not sure how to go about this.

EDIT : I tried doing this

I created a file and push the hash in it by doing this

echo "2c1400f69867571ab4e60d3b8f01e0b17c7be89e321f91f8a07cd39eeba202e2" >file.txt

then followed by this

openssl rsautl -inkey mykey.key -pubin -encrypt -in file.txt > file.txt.enc

when I tried to open file.txt.enc on notepad, i got this:

ciphertext output

Not sure if this is a successful encryption as I need the ciphertext result in binary format.

  • That looks pretty binary to me, but notepad isn't a good choice to show binary data, use a hex editor. Your input on the other hand is a hex string interpreted as bytes instead of proper binary data, so you probably won't get the expected result. – CodesInChaos Sep 30 '15 at 15:32
  • Unless you need to be compatible with legacy software, make sure you're using OAEP padding, it's much more secure than the older PKCS#1v1.5 padding or textbook RSA. – CodesInChaos Sep 30 '15 at 15:33
  • Note that hashes almost never need to be secret so encrypting them rarely makes sense. Are you sure someone doesn't want you to sign the hash? RSA signing is often described as "encrypting with the private key" by people who know only vaguely what they're talking about -- but see numerous answers on security.SE or crypto.SE why that's not correct. – dave_thompson_085 Sep 30 '15 at 19:26


$ cat file.txt | openssl rsautl -encrypt -pubin -inkey mykey.key > secret.txt
cat secret.txt
# more binary data here...

and then:

$ cat secret.txt | openssl rsautl -decrypt -inkey myprivatekey.key

Both keys should be in PEM format.


Not sure if this is a successful encryption as i need the ciphertext result in binary format.

Well, if you really want to test if the encryption is successful, try immediately decrypting it and see if you get your plaintext back. If you get your plaintext back, you are good.

I think you are confused on what "binary format" actually means.



is actually a textual hex format - a text representation of a binary value.

Any file or chunk of data on a system is ultimately a series of bytes - and a byte is (to keep it simple, don't worry about bits yet) simply a number from the range 0 to 255. You can write or display a byte as a decimal number, hexadecimal number, binary number, or anything required by the application processing the bytes.

There are various schemes to represent text as numbers, UTF-8, Unicode, etc.

To illustrate, the string of text "2c1400f69867571ab4e60d3b8f01e0b17c7be89e321f91f8a07cd39eeba202e2" is the following when converted to bytes (shown in decimal):

 50  99  49  52  48  48 102  54  57  56  54  55  53  55  49  97
 98  52 101  54  48 100  51  98  56 102  48  49 101  48  98  49
 55  99  55  98 101  56  57 101  51  50  49 102  57  49 102  56
 97  48  55  99 100  51  57 101 101  98  97  50  48  50 101  50

plus a newline. A total of 65 bytes, 130 if your locale uses 16-bit characters e.g. Unicode. Unicode will be the same values as above except with a 0 in front of each value. It's not equivalent to the value the text represents, obviously.

So, the above series of bytes is what you told openssl to actually encrypt, not the actual value "2c1400f69867571ab4e60d3b8f01e0b17c7be89e321f91f8a07cd39eeba202e2" (which would be an insanely long decimal number).

So, based on above, your file.txt.enc likely consists of 65 or 130 random values. That's what Notepad is trying to render.

Now, if you want the text hex representation of the contents of your file.txt.enc - I think this will work (based on this):

xxd -p file | tr -d '\n'

  • 1
    You're right about the input but not the output. RSA does not encrypt each input byte to an output byte like RC4 or AES-CTR does; it takes a block of bytes (or bits) up to the "size" of the key (more exactly, the size of the modulus in the key, nowadays usually 2048 bits = 256 bytes although in past years smaller keys were used) less a margin that allows padding needed for security (usually 10 to 20 bytes), and encrypts that block to a result the same size as the key/modulus; decrypting removes the padding to give the variable-length input. – dave_thompson_085 Sep 30 '15 at 19:21
  • Thanks for explaining everything guys, I was confused by the term binary data, I'm still trying to understand public key cryptography properly. – Jeevan Daniel Mahtani Oct 1 '15 at 0:47

You will find the answers to many of your questions (with examples) here:


In particular, if you are sending the encrypted result from your procedure via email, you might wish to base64 encode it using either the '-a' option in 'enc' subcommand, or as a separate step, the 'base64' openssl subcommand.

This will then produce a result that is ASCII text rather than a binary file of unprintable bytes.

I would suggest that you check your encryption by first of all, creating your own (test) RSA keyfile (also using openssl, see link above), and then trying the 'round trip' of encrypting-(base64)encoding-decoding-decrypting.

If your result using your own test RSA key (pair) is the same as what you put in, then it's reasonable to say that at least you have the mechanics all correct.

You could then try sending some other text to your intended recipient using the supplied RSA key file to verify all is correct, prior to sending your "real" data - which I hope is not the same as you have typed in your question ! (unless you are just using encryption for authentication).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.