3

I'm having trouble getting a string tool to provide what I need (due to my ignorance). I have a string based on CPU features. The string will change as different processors offer different features:

# Example from a modern Core i5 4th gen
SUNCC_CXXFLAGS="-D__SSE2__ -D__SSE3__ -D__SSSE3__ -D__SSE4_1__ -D__SSE4_2__ -D__AES__ -D__PCLMUL__ __AVX__ ..."

The string works great for Sun Studio 12.3 and above. For Sun Studio 12.2 and below, I can only use SSE2, SSE3, SSSE3, SSE4.1 and SSE4.2. AES and above defines cause an obscure error, so they must be filtered out of the flags.

Put another way I need the intersection of the two sets:

# Cannot use AES and above for SunCC 12.2
ALLOWED_CXXFLAGS="-D__SSE2__ -D__SSE3__ -D__SSSE3__ -D__SSE4_1__ -D__SSE4_2__"
# New processor, needs to be filtered due to old compiler
SUNCC_CXXFLAGS="-D__SSE2__ -D__SSE3__ -D__SSSE3__ -D__SSE4_1__ -D__SSE4_2__ -D__AES__ -D__PCLMUL__ __AVX__ ..."

I've seen a number of questions and answers with awk matching (and non-matching) regexes and lines. But I need to filter based on tokens in a single line.

I tried the following, which does not produce expected results:

$ echo "-D__SSE2__ -D__SSE3__ -D__SSSE3__ -D__SSE4_1__ -D__SSE4_2__ -D__AES__ -D__PCLMUL__" | \
  nawk '!/(-D__SSE2__|-D__SSE3__|-D__SSSE3__)/'
$

Another twist: this is Solaris, so the tools don't have a lot of the options present in GNU tools. Its one of the reasons I tried awk instead of sed or grep.

How can I filter out the tokens that don't match my set of tokens?

  • 1
    Your expression prints non-matching records (lines) rather than non-matching substrings within those records - you could try nawk '{gsub(/-D__SSE2__|-D__SSE3__|-D__SSSE3__/,"")}1' but I would think it's just as likely that your version of sed supports a BRE of the form sed 's/-D__SSE2__\|-D__SSE3__\|-D__SSSE3__//g' – steeldriver Jul 15 '16 at 21:08
  • @John1024 - desired output only includes SSE. Modern processors will have all the SSE defines (plus additional defines which are causing problems). Older platforms, like Pentium Pro or Pentium II, may be missing SSE3 and SSSE3. In the case of older Pentiums, its conceivable the remaining CXXFLAGS only includes SSE2. – user56041 Jul 15 '16 at 21:52
  • @jww Very good. I have updated the answer accordingly. – John1024 Jul 15 '16 at 22:25
7
+50

To select only the SSE flags, try:

awk '/SSE/' ORS=' ' RS=' '

The key thing here is setting the record separators on input and output to a space. That way, each option is accepted or rejected separately.

For example:

$ SUNCC_CXXFLAGS="-D__SSE2__ -D__SSE3__ -D__SSSE3__ -D__SSE4_1__ -D__SSE4_2__ -D__AES__ -D__PCLMUL__ ..."
$ newFLAGS="$(echo "$SUNCC_CXXFLAGS" | awk '/SSE/' ORS=' ' RS=' ')"
$ echo "$newFLAGS"
-D__SSE2__ -D__SSE3__ -D__SSSE3__ -D__SSE4_1__ -D__SSE4_2__ 

SSE appears to be a tight enough match here. If it isn't, we can be more specific:

$ newFLAGS="$(echo "$SUNCC_CXXFLAGS" | awk '/^-D__(SSE2|SSE3|SSSE3|SSE4.1|SSE4.2)__/' ORS=' ' RS=' ')"
$ echo "$newFLAGS"
-D__SSE2__ -D__SSE3__ -D__SSSE3__ -D__SSE4_1__ -D__SSE4_2__ 

Alternative: excluding SSE and AES

$ echo "$SUNCC_CXXFLAGS" | nawk '!/SSE|AES/' ORS=' ' RS=' '
-D__PCLMUL__ ...

Keeping options that match SSE or sse

$ SUNCC_CXXFLAGS="-D__SSE2__ -D__SSE3__ -D__SSSE3__ -D__SSE4_1__ -D__SSE4_2__ -D__AES__ -D__PCLMUL__ -xarch=sse3"
$ newFLAGS="$(echo "$SUNCC_CXXFLAGS" | awk '/SSE|sse/' ORS=' ' RS=' ')"
$ echo "$newFLAGS"
-D__SSE2__ -D__SSE3__ -D__SSSE3__ -D__SSE4_1__ -D__SSE4_2__ -xarch=sse3

The change here is that we replaced the regex /SSE/ with /SSE|sse/. Because the vertical bar, |, means logical-or, this matches either SSE or sse.

  • Perfect, thank you very much. It tested as expected, and was checked in. – user56041 Jul 15 '16 at 21:27
  • Very good. Glad it worked. – John1024 Jul 15 '16 at 21:29
  • Sorry to dig up an old thread. Solaris is acting up again. We found we needed to add -xarch to fix another problem, which means we need to match a lowercase sse too. echo "$newFLAGS" should return -D__SSE2__ -D__SSE3__ -D__SSSE3__ -D__SSE4_1__ -D__SSE4_2__ -xarch=ssse3. I visited 3.8 Case Sensitivity in Matching of the manual, but its not quite clicking for me. Would you be able to modify to match lowercase, too (but preserve case on output)? Or do you want a new question? – user56041 Sep 12 '16 at 8:12
  • 1
    @jww I just added a new section to the end of the answer that does what I think you are asking for. (Because you are on Solaris, try to use nawk or better yet /usr/xpg4/bin/awk or /usr/xpg6/bin/awk in place of Solaris' awk.) – John1024 Sep 12 '16 at 17:40
  • @jww That was kind and generous of you! I'm glad that the answer helped. – John1024 Sep 14 '16 at 2:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy