1

I performed a clone of Zcash:

$ git clone https://github.com/zcash/zcash.git
$ cd zcash/
$ git checkout v1.0.1
$ ./zcutil/fetch-params.sh

I'm trying to run fetch-params.sh on both OS X and Solaris. Its a Linux script, so it needs a little messaging. Below is the immediate diff; and the complete script with changes is at the end of the question.

diff --git a/zcutil/fetch-params.sh b/zcutil/fetch-params.sh
index ac5327b..e2e9807 100755
--- a/zcutil/fetch-params.sh
+++ b/zcutil/fetch-params.sh
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash

 set -eu

@@ -12,6 +12,9 @@ SPROUT_VKEY_URL="https://z.cash/downloads/$SPROUT_VKEY_NAME"
 SHA256CMD="$(command -v sha256sum || echo shasum)"
 SHA256ARGS="$(command -v sha256sum >/dev/null || echo '-a 256')"

+IS_DARWIN=$(uname -s | grep -i -c darwin)
+IS_SOLARIS=$(uname -s | grep -i -c sunos)
+
 function fetch_params {
     local url="$1"
     local output="$2"
@@ -45,13 +48,20 @@ EOF

 # Use flock to prevent parallel execution.
 function lock() {
-    local lockfile=/tmp/fetch_params.lock
-    # create lock file
-    eval "exec 200>/$lockfile"
-    # acquire the lock
-    flock -n 200 \
-        && return 0 \
-        || return 1
+   local lockfile="/tmp/fetch_params.lock"
+    if [[ ("$IS_SOLARIS" -ne "0" || "$IS_DARWIN" -ne "0") ]]; then
+        # http://unix.stackexchange.com/a/13025
+        mkdir "$lockfile" \
+            && return 0 \
+            || return 1
+    else
+        # create lock file
+        eval "exec 200>/$lockfile"
+        # acquire the lock
+        flock -n 200 \
+           && return 0 \
+           || return 1
+    fi
 }

 function exit_locked_error {
@@ -105,5 +115,11 @@ EOF
 }

 main
-rm -f /tmp/fetch_params.lock
+
+if [[ ("$IS_SOLARIS" -ne "0" || "$IS_DARWIN" -ne "0") ]]; then
+    rm -rf /tmp/fetch_params.lock
+else
+    rm -f /tmp/fetch_params.lock
+fi
+
 exit 0

The problem I am having is main is not executing. The script quietly exits after the IS_SOLARIS=$(uname -s | grep -i -c sunos):

riemann:zcash$  bash -x ./zcutil/fetch-params.sh 
+ set -eu
...
++ uname -s
++ grep -i -c darwin
+ IS_DARWIN=1
++ uname -s
++ grep -i -c sunos
+ IS_SOLARIS=0
riemann:zcash$ 

Here's the weird thing... It happens on both OS X and Solaris. And if I set change IS_SOLARIS=$(uname -s | grep -i -c sunos) to IS_SOLARIS=0 on OS X, then it works. Or if I comment out set -eu then it works. I've never used set -eu before, but I don't understand its [possible] failure given the explanation at When to use set -e.

OS X provides GNU bash, version 3.2.53; while Solaris 11 provides GNU bash, version 4.1.17.

Please forgive my ignorance... Why does the test for IS_DARWIN and IS_SOLARIS cause the script to die? Why is it not executing main?


Full Modified Script

#!/usr/bin/env bash

set -eu

PARAMS_DIR="$HOME/.zcash-params"

SPROUT_PKEY_NAME='sprout-proving.key'
SPROUT_VKEY_NAME='sprout-verifying.key'
SPROUT_PKEY_URL="https://z.cash/downloads/$SPROUT_PKEY_NAME"
SPROUT_VKEY_URL="https://z.cash/downloads/$SPROUT_VKEY_NAME"

SHA256CMD="$(command -v sha256sum || echo shasum)"
SHA256ARGS="$(command -v sha256sum >/dev/null || echo '-a 256')"

IS_DARWIN=$(uname -s | grep -i -c darwin)
IS_SOLARIS=$(uname -s | grep -i -c sunos)

function fetch_params {
    local url="$1"
    local output="$2"
    local dlname="${output}.dl"
    local expectedhash="$3"

    if ! [ -f "$output" ]
    then
        echo "Retrieving: $url"
        wget \
            --progress=dot:giga \
            --output-document="$dlname" \
            --continue \
            --retry-connrefused --waitretry=3 --timeout=30 \
            "$url"

        "$SHA256CMD" $SHA256ARGS --check <<EOF
$expectedhash  $dlname
EOF

        # Check the exit code of the shasum command:
        CHECKSUM_RESULT=$?
        if [ $CHECKSUM_RESULT -eq 0 ]; then
            mv -v "$dlname" "$output"
        else
           echo "Failed to verify parameter checksums!"
           exit 1
        fi
    fi
}

# Use flock to prevent parallel execution.
function lock() {
    local lockfile="/tmp/fetch_params.lock"
    if [[ ("$IS_SOLARIS" -ne "0" || "$IS_DARWIN" -ne "0") ]]; then
        # http://unix.stackexchange.com/a/13025
        mkdir "$lockfile" \
            && return 0 \
            || return 1
    else
        # create lock file
        eval "exec 200>/$lockfile"
        # acquire the lock
        flock -n 200 \
           && return 0 \
           || return 1
    fi
}

function exit_locked_error {
    echo "Only one instance of fetch-params.sh can be run at a time." >&2
    exit 1
}

function main() {

    lock fetch-params.sh \
    || exit_locked_error

    cat <<EOF
Zcash - fetch-params.sh

This script will fetch the Zcash zkSNARK parameters and verify their
integrity with sha256sum.

If they already exist locally, it will exit now and do nothing else.
EOF

    # Now create PARAMS_DIR and insert a README if necessary:
    if ! [ -d "$PARAMS_DIR" ]
    then
        mkdir -p "$PARAMS_DIR"
        README_PATH="$PARAMS_DIR/README"
        cat >> "$README_PATH" <<EOF
This directory stores common Zcash zkSNARK parameters. Note that it is
distinct from the daemon's -datadir argument because the parameters are
large and may be shared across multiple distinct -datadir's such as when
setting up test networks.
EOF

        # This may be the first time the user's run this script, so give
        # them some info, especially about bandwidth usage:
        cat <<EOF
The parameters are currently just under 911MB in size, so plan accordingly
for your bandwidth constraints. If the files are already present and
have the correct sha256sum, no networking is used.

Creating params directory. For details about this directory, see:
$README_PATH

EOF
    fi

    cd "$PARAMS_DIR"

    fetch_params "$SPROUT_PKEY_URL" "$PARAMS_DIR/$SPROUT_PKEY_NAME" "8bc20a7f013b2b58970cddd2e7ea028975c88ae7ceb9259a5344a16bc2c0eef7"
    fetch_params "$SPROUT_VKEY_URL" "$PARAMS_DIR/$SPROUT_VKEY_NAME" "4bd498dae0aacfd8e98dc306338d017d9c08dd0918ead18172bd0aec2fc5df82"
}

main

if [[ ("$IS_SOLARIS" -ne "0" || "$IS_DARWIN" -ne "0") ]]; then
    rm -rf /tmp/fetch_params.lock
else
    rm -f /tmp/fetch_params.lock
fi

exit 0
  • 8
    Because grep exits with non-zero status, and the script runs with set -e? – steeldriver Nov 9 '16 at 19:12
  • Thanks Steeldriver. As far as I know, grep exits with 0 even when it fails to match. See, for example, Why the exit command of grep is 0 even if a match is not found? I'm trying to find a better reference/man page entry. – user56041 Nov 9 '16 at 19:19
  • 1
    Compare bash -c 'var=$(echo "foo" | grep -c "foo"); echo $?' and bash -c 'var=$(echo "foo" | grep -c "bar"); echo $?' – steeldriver Nov 9 '16 at 19:24
  • 3
    jww, you're aware of the typo/confusion in that linked answer? – Jeff Schaller Nov 9 '16 at 19:25
3

You can just use the Bash pattern matching like this.

[[ $(uname) = *[Dd]arwin* ]]; IS_DARWIN=$?
[[ $(uname) = *[Ss]un[Oo][Ss]* ]];  IS_SOLARIS=$?

This is an extended test builtin to Bash. To cut down on the calls to uname save the value to a variable.

uname=$(uname)
[[ $uname = *[Dd]arwin* ]]; IS_DARWIN=$?
[[ $uname = *[Ss]un[Oo][Ss]* ]];  IS_SOLARIS=$?

This can be further simplified as:

uname=$(uname)
[[ $uname =~ [Dd]arwin ]]; IS_DARWIN=$?
[[ $uname =~ [Ss]un[Oo][Ss] ]];  IS_SOLARIS=$?

Or as:

uname=$(uname | tr [A-Z] [a-z])
[[ $uname =~ darwin ]]; IS_DARWIN=$?
[[ $uname =~ sunos ]];  IS_SOLARIS=$?

Or simplest

uname=$(uname)
[[ $uname =~ Darwin ]]; IS_DARWIN=$?
[[ $uname =~ SunOS ]];  IS_SOLARIS=$?

Example runs from Solaris (SunOS).

-bash-3.2$ uname
SunOS
-bash-3.2$ uname=$(uname | tr [A-Z] [a-z])
-bash-3.2$ [[ $uname =~ sunos ]];  IS_SOLARIS=$?; echo $IS_SOLARIS
0

-bash-3.2$ uname
SunOS
-bash-3.2$ uname=$(uname)
-bash-3.2$ [[ $uname =~ [Ss]un[Oo][Ss] ]];  IS_SOLARIS=$?; echo $IS_SOLARIS
0

-bash-3.2$ uname
SunOS
-bash-3.2$ uname=$(uname)
-bash-3.2$ [[ $uname =~ SunOS ]];  IS_SOLARIS=$?; echo $IS_SOLARIS
0
-bash-3.2$

-bash-3.2$ uname
SunOS
-bash-3.2$ [[ $(uname) =~ SunOS ]];  IS_SOLARIS=$?; echo $IS_SOLARIS
0
-bash-3.2$
  • This produces incorrect results: [[ $uname =~ [Dd]arwin ]]; IS_DARWIN=$?; echo $IS_DARWIN. It always prints 1, even on Solaris. It also prints 1 if I omit the uname=$(uname). – user56041 Nov 10 '16 at 12:57
  • 1
    Exit code 1 (and higher) is boolean false (EXIT_FAILURE constant in C). Exit code 0 is boolean true (EXIT_SUCCESS constant in C). This answer could be fixed by putting a ! (logical negation) in front of the [[. – sourcejedi Nov 10 '16 at 13:31
  • You need to run the uname=$(uname) line first. See updated examples from Solaris – BillThor Nov 10 '16 at 21:10
  • You need to run the uname=$(uname) line first . See the updates with examples from Solaris. – BillThor Nov 10 '16 at 21:15
1

$ man grep

/EXIT

EXIT STATUS
   Normally the exit status is 0 if a line is selected, 1 if no lines were
   selected, and 2 if an error occurred.  However, if the -q or --quiet or
   --silent  is  used and a line is selected, the exit status is 0 even if
   an error occurred.

EDIT: hopefully this makes it clear. I know it's a surprise if you weren't looking for this behaviour.

This can be verified by referring to the secret POSIX standard (seriously, they slightly obscure the webpage hoping more people buy print versions).

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/grep.html#tag_20_55_14

EXIT STATUS

The following exit values shall be returned:

 0
    One or more lines were selected.
 1
    No lines were selected.
>1
    An error occurred.

example solution

Albeit without trying to detect when "an error occurred":

if (uname -s | grep -i darwin >/dev/null); then
  IS_DARWIN=1
fi
  • Thanks @sourcejedi . What does "...if a line is selected" mean exactly, and how does it interact with -c? The best I can tell, there are no errors, and either 0 or 1 shows up as expected. – user56041 Nov 9 '16 at 19:36
  • 2
    selected = matches the regular expression; -c is just counting the matches -- it doesn't change the return code – Jeff Schaller Nov 9 '16 at 19:58
  • OK, thanks @sourcejedi. I have to be careful about how I proceed because I work on a lot of systems, like BSDs, Solaris, Linux, OS X, etc. I need to try to stay very close to Posix. Is it possible to disable regular expression matching, and just match the token as a substring like 'Darwin'? It looks like -F is Posix and may do that, but I get the same result. I also dropped the -i and performed a comparison for Darwin and SunOS with no joy. – user56041 Nov 9 '16 at 20:27
  • sorry, you're not making any sense to me. the example solution should be fully portable, right? – sourcejedi Nov 9 '16 at 20:52
  • @sourcejedi - "you're not making any sense to me" - Sorry about that. " the example solution should be fully portable, right..." - I don't know. I'm a C/C++ guy. I struggle with scripting at times. – user56041 Nov 10 '16 at 12:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy