1

This is kind of a continuation of Awk and discarding non-matching tokens in a string? I am using Awk to filter unwanted compiler flags derived from CPU features (SunCC 12.2 cannot handle the same flags as SunCC 12.5). The CXXFLAGS produced by the filter results in a compile error under SunCC 12.2:

/opt/solstudio12.2/bin/CC -DDEBUG -g -xO0 -D__SSE2__ -D__SSE3__ -D__SSSE3__
usage: usage: CCCC [ options ] files.  Use 'CC -flags' for details
 [ options ] files.  Use 'CC -flags' for details
usage: CC [ options ] files.  Use 'CC -flags' for details
usage: CC [ options ] files.  Use 'CC -flags' for details
m64 -KPIC -template=no%extdef -w -erroff=wvarhidemem -erroff=voidretw -c cryptlib.cpp

What is happening is, Awk is producing -D__SSE2__ -D__SSE3__ -D__SSSE3__ + \n. So the CXXLAGS is being broken into two lines, which leads to the following from GNUmake:

  1. /opt/solstudio12.2/bin/CC -DDEBUG -g -xO0 -D__SSE2__ -D__SSE3__ -D__SSSE3__
  2. -m64 -KPIC -template=no%extdef -w -erroff=wvarhidemem -erroff=voidretw -c cryptlib.cpp

When I debug the Bash script that drives this. Lines 1012 and 1014 appear to be problematic if I am parsing the output correctly.

$ PS4='Line ${LINENO}: ' bash -x ./cryptest.sh
...
Line 1005: SUNCC_CXXFLAGS='-D__SSE2__ -D__SSE3__ -D__SSSE3__'
Line 1008: echo 'PLATFORM_CXXFLAGS: -D__SSE2__' -D__SSE3__ -D__SSSE3__
PLATFORM_CXXFLAGS: -D__SSE2__ -D__SSE3__ -D__SSSE3__
Line 1009: echo 'SUNCC_CXXFLAGS: -D__SSE2__ -D__SSE3__ -D__SSSE3__'
SUNCC_CXXFLAGS: -D__SSE2__ -D__SSE3__ -D__SSSE3__
LLine 1012: echo '-D__SSE2__ -D__SSE3__ -D__SSSE3__'
LLine 1012: /usr/gnu/bin/awk /SSE/ 'ORS= ' 'RS= '
Line 1012: SUNCC_SSE_CXXFLAGS='-D__SSE2__ -D__SSE3__ -D__SSSE3__
 '
Line 1014: echo 'SUNCC_SSE_CXXFLAGS: -D__SSE2__ -D__SSE3__ -D__SSSE3__
 '
SUNCC_SSE_CXXFLAGS: -D__SSE2__ -D__SSE3__ -D__SSSE3__

Line 1020: [[ -e /opt/solstudio12.2/bin/CC ]]

Here's the corresponding Bash source. I believe the offender is line 1012, SUNCC_SSE_CXXFLAGS=$(echo "$SUNCC_CXXFLAGS" | "$AWK" '/SSE/' ORS=' ' RS=' ').

1003   # If PLATFORM_CXXFLAGS is for SunCC, then use them
1004   if [[ ("$SUNCC_121_OR_ABOVE" -ne "0") ]]; then
1005       SUNCC_CXXFLAGS="${PLATFORM_CXXFLAGS[@]}"
1006   fi
1007   
1008   echo "PLATFORM_CXXFLAGS: ${PLATFORM_CXXFLAGS[@]}"
1009   echo "SUNCC_CXXFLAGS: ${SUNCC_CXXFLAGS[@]}"
1010
1011   # Sun Studio 12.3 and below workaround, http://github.com/weidai11/cryptopp/issues/228
1012   SUNCC_SSE_CXXFLAGS=$(echo "$SUNCC_CXXFLAGS" | "$AWK" '/SSE/' ORS=' ' RS=' ')
1013
1014   echo "SUNCC_SSE_CXXFLAGS: ${SUNCC_SSE_CXXFLAGS[@]}"
...
1018   ############################################
1019   # Sun Studio 12.2
1020   if [[ (-e "/opt/solstudio12.2/bin/CC") ]]; then

My question is, how do I suppress new line when Awk'ing an array under Solaris?


Here's how the script invokes make. The script can be found at cryptest.sh. Its a monster, but the problematic lines are around 2880 (line 1000 from above is the reduced test case to help isolate things).

CXXFLAGS="-DDEBUG -g -xO0 ${SUNCC_SSE_CXXFLAGS[@]}"
CXX="/opt/solstudio12.2/bin/CC" CXXFLAGS="$CXXFLAGS" "$MAKE" "${MAKEARGS[@]}" static dynamic cryptest.exe 2>&1 | tee -a "$TEST_RESULTS"

Here's how PLATFORM_CXXFLAGS is formed. It is necessary for our source code because code paths are activate based on GCC defines (but SunCC does not define them).

if [[ ("$IS_SOLARIS" -ne "0") && ("$SUNCC_121_OR_ABOVE" -ne "0") ]]; then
    ISAINFO=$(isainfo -v 2>/dev/null)
    if [[ ($(echo "$ISAINFO" | "$GREP" -c "sse2") -ne "0") ]]; then PLATFORM_CXXFLAGS+=("-D__SSE2__"); fi
    if [[ ($(echo "$ISAINFO" | "$GREP" -c "sse3") -ne "0") ]]; then PLATFORM_CXXFLAGS+=("-D__SSE3__"); fi
    if [[ ($(echo "$ISAINFO" | "$GREP" -c "ssse3") -ne "0") ]]; then PLATFORM_CXXFLAGS+=("-D__SSSE3__"); fi
    if [[ ($(echo "$ISAINFO" | "$GREP" -c "sse4.1") -ne "0") ]]; then PLATFORM_CXXFLAGS+=("-D__SSE4_1__"); fi
    if [[ ($(echo "$ISAINFO" | "$GREP" -c "sse4.2") -ne "0") ]]; then PLATFORM_CXXFLAGS+=("-D__SSE4_2__"); fi
    if [[ ($(echo "$ISAINFO" | "$GREP" -c "aes") -ne "0") ]]; then PLATFORM_CXXFLAGS+=("-D__AES__"); fi
    if [[ ($(echo "$ISAINFO" | "$GREP" -c "pclmulqdq") -ne "0") ]]; then PLATFORM_CXXFLAGS+=("-D__PCLMUL__"); fi
    if [[ ($(echo "$ISAINFO" | "$GREP" -c "rdrand") -ne "0") ]]; then PLATFORM_CXXFLAGS+=("-D__RDRND__"); fi
    if [[ ($(echo "$ISAINFO" | "$GREP" -c "rdseed") -ne "0") ]]; then PLATFORM_CXXFLAGS+=("-D__RDSEED__"); fi
    if [[ ($(echo "$ISAINFO" | "$GREP" -c "avx") -ne "0") ]]; then PLATFORM_CXXFLAGS+=("-D__AVX__"); fi
    if [[ ($(echo "$ISAINFO" | "$GREP" -c "avx2") -ne "0") ]]; then PLATFORM_CXXFLAGS+=("-D__AVX2__"); fi
    if [[ ($(echo "$ISAINFO" | "$GREP" -c "bmi") -ne "0") ]]; then PLATFORM_CXXFLAGS+=("-D__BMI__"); fi
    if [[ ($(echo "$ISAINFO" | "$GREP" -c "bmi2") -ne "0") ]]; then PLATFORM_CXXFLAGS+=("-D__BMI2__"); fi
fi
2
  • May we see how you're actually using SUNCC_SSE_CXXFLAGS in your CC command? You seem to be assigning a string variable via command substitution i.e. SUNCC_SSE_CXXFLAGS=$( awk command ) but expecting to get an array ${SUNCC_SSE_CXXFLAGS[@]} Aug 24, 2016 at 1:58
  • @steeldriver - OK done. I added the invocation of make, and linked back to the actual script. And yes, I could be incorrectly using the arrays. I expect array + operation = array, but that may not hold. I switched to arrays because things did not work as expected when make's invocation was $MAKE $MAKEARGS. $MAKEARGS was being interpreted as commands. (The underlying problem is I am a C/C++ guy; and not a Bash guy. So I cobble these scripts together).
    – user56041
    Aug 24, 2016 at 2:05

1 Answer 1

3

In:

echo '-D__SSE2__ -D__SSE3__ -D__SSSE3__' | awk /SSE/ RS=' ' ORS=' '

You're feeding -D__SSE2__ -D__SSE3__ -D__SSSE3__<newline> to awk (echo adds the newline).

With RS=' ', that means 3 records: -D__SSE2__, -D__SSE3__ and -D__SSSE3__<newline>. All match /SSE/ so are all printed followed by the output record separator (ORS=' '). For the 3rd one, that's -D__SSSE3__<newline><space>

So here, you want either to not output the <newline> in the first place:

printf %s '-D__SSE2__ -D__SSE3__ -D__SSSE3__' | awk...

Or add an extra space, so the <newline> be on its own record (that doesn't match /SSE/):

echo '-D__SSE2__ -D__SSE3__ -D__SSSE3__ ' | awk...

Or, since you're using GNU awk as opposed to the Solaris one, make any sequence of spacing character the record separator:

echo -D__SSE2__ -D__SSE3__ -D__SSSE3__ | gawk -v 'RS=[[:space:]]+' ...
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy