2

I'm learning about SHA1 (specifically wrt Git), and I wanted to sanity-check my understanding by calculating a string's SHA1 with different methods - I expected identical SHA1 hashes, but instead I got distinct results from three of four methods:

>git hash-object --stdin <<< "Apple Pie"
23991897e13e47ed0adb91a0082c31c82fe0cbe5

.

>sha1sum <<< "blob 9\0Apple Pie"
332cd56150dc8b954c0b859bd4aa6092beafa00f  -

.

>printf 'blob 9\0Apple Pie' > foo.txt
>sha1sum foo.txt
9eed377bbdeb4aa5d14f8df9cd50fed042f41023  foo.txt

.

>openssl sha1 foo.txt
SHA1(foo.txt)= 9eed377bbdeb4aa5d14f8df9cd50fed042f41023

The accepted answer to this Stack Overflow question says that git hash-object runs a SHA1 hash on the specified content prefixed with "blob [file size]/0". Thus I explicitly prefixed that text to the strings I tested with the non-git method.

Why all these different results? I thought SHA1 was a specific and unique hash of a given string, and that there were not different "types" of SHA1 - is that not true?

2
  • 3
    Use hexdump -C instead of sha1sum and you'll see the there's already a difference in your input. May 14 '17 at 20:57
  • 1
    ...and printf 'blob 10\0Apple Pie\n' | sha1sum happens to be what git reports. May 14 '17 at 21:04
4

The differences don't come from SHA1, but the input. The here-string syntax appends a newline, as we can see with od:

$ od -c <<< foo
0000000   f   o   o  \n

So in your git command the input is the ten characters Apple Pie\n.

In addition, the double quotes you used in the here-strings don't support backslash escapes like \n or \nnn, so <<< "blob 9\0Apple Pie" gives a string containing a literal backslash and a zero.

printf however does interpret \0 as the NUL byte, and it doesn't add a trailing newline, so with the newline added and the length fixed, we should get the expected output:

$ printf 'blob 10\0Apple Pie\n' | sha1sum
23991897e13e47ed0adb91a0082c31c82fe0cbe5  -

We could try to do the same with the here-string using the $'' quote which does support \0 as representing the NUL byte, but that may not work in all shells, since the NUL byte ends the string. E.g. Bash cannot deal with it, zsh can:

$ zsh -c "sha1sum <<< $'blob 10\0Apple Pie'"
23991897e13e47ed0adb91a0082c31c82fe0cbe5  -
1
  • String-handling in Bash continues to flummox me. Thank you for both answering the question and shedding a little bit more light on Bash string-handling.
    – StoneThrow
    May 15 '17 at 5:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.