I'm using a long gawk 3.1.6 script to do a complex conversion of Zim markdown text into GtkDialog code and am stuck on the following problem...

Sample ASCII input...

[[link|label label]] [[link]] @tag more text

Commandline test to find right regex...

re="[][][][]"; echo '[[link|label label]] [[link]] @tag more text' | awk -v RE=$re '{split($0,A,RE); printf "\n(" A[1] ")(" A[2] ")(" A[3] ")(" A[4] ")(" A[5] ")(" A[6] ")(" A[7] ")(" A[8] ")\n"}'

The regex "[][][][]" splits out the two hyperlink forms quite nicely so that's not a problem.

It would be more understandable if we could divided it in two -- "[][]" and "[][]". We are looking for either "[[" or "]]" to split on. The order of the characters in the class have to be reversed to comply with class meta-character restrictions.

The problem is in also splitting out the "@tag" into just "tag". "tag" could be any alphanumeric text either followed by a space or the end of the string.

Executing the commandline test above yields...

()(link|label label)( )(link)( @tag more text)()()

But I need it to yield...

()(link|label label)( )(link)( )(tag)(more text)

I've tried numerous regex strings like "[][][][]|@[[:alnum:]]*" which drops the entire word and yields...

()(link|label label)( )(link)( )( more text)()

and "[][][][]|@" which yields...

()(link|label label)( )(link)( )(tag more text)()

Any ideas?

  • Is changing FS an option or is that impossible within your larger script? – terdon Jan 15 '14 at 19:24
  • Not possible as this happens in an awk function that's part of a 760 line awk program. – DocSalvager Jan 16 '14 at 8:53

I don't think you can do this in a single regex, but since you're using gawk, you can use some gawk extensions:

awk '{
    n = split($0, a, /\[\[|\]\]|@[[:alnum:]]+/, s)
    for (i=1; i<=n; i++) {
        printf "(%s)", a[i]
        if (match(s[i], /^@(.+)/, m))
            printf "(%s)", m[1]
    print ""
}' <<END
[[link|label label]] [[link]] @tag more text
some text with @anothertag and [[another|link]]
()(link|label label)( )(link)( )(tag)( more text)
(some text with )(anothertag)( and )(another|link)()
| improve this answer | |
  • Unfortunately, the 4th argument to split() that this depends on is not supported in 3.1.6. However, I've written a function to use in place of split() that does. Won't be as fast as the builtin, but awk is so fast anyhow it doesn't look to be a problem. Still testing. Will post when perfected. Many thanks! – DocSalvager Jan 17 '14 at 9:41

This is ugly and horrible but provides the desired output:

$ echo '[[link|label label]] [[link]] @tag more text' | 
 awk -vFS="[\\\\[\\\\] @]" '{
  printf "\n(" $1 ")(" $3" "$4 ")(" $5 ")(" $9 ")(" $10 ")(" $13 ")("; 
  for(i=15;i<=NF;i++){rest=rest" "$(i)}
  printf "%s)\n", rest;

()(link|label label)()(link)()(tag)(more text)

The trick is setting the field separator to any of [,],@ or (space). If you can do so within your program, it should give the desired output.

| improve this answer | |

Implemented Solution

This is, in effect, an implementation of glenn's answer with the addition of the user-defined function stringSplit() to use instead of the Gawk 3.1.6 builtin split() function which does not support the optional 4th argument ([seps], an array to hold seperators) that we need. Gawk 3.1.6 does support the similar purpose optional 3rd argument to match() needed, but [seps] is not available till Gawk 4.0.0.

# stringSplit(str,fld,rx,[sep])
#     Split string on regex delimeter preserving regex-seperators. Gawk 3.1.6
#     equivalent to builtin split() function of later versions which add
#     support for an optional 4th argument, ([seps]), an array to hold the
#     evaluated regular-expressions.
# Arguments:
#     str
#       string to split
#     fld
#       array of the resulting fields
#     rx
#       regular expression (regex) to split on
#     [sep]
#       optional array of seperator strings matching the regex
# Revised:
#     20140117 docsalvage
function stringSplit(str,fld,rx,sep,    searchstr,searchndx,match1,matchn,matches) {
  searchstr = str                     # copy of str to use in while() loop
  searchndx = match(searchstr, rx)    # index in searchstr where rx(regex) found
  match1    = searchndx               # preserve result of first match attempt
  matchn    = 1                       # match number (index in array of matches)
  matches   = 0                       # number of matches returned by split()
  while (RLENGTH > 0) {               # more reliable than while(searchndx > 0)
    # save match
    sep[matchn] = substr(searchstr, searchndx, RLENGTH)
    # match() only searches from beginning so give it just remainder of str
    searchstr   = substr(searchstr, searchndx + RLENGTH)
    # printf("sep[%2d]: %s, searchndx: %2d, RLENGTH: %2d, searchstr: %s\n", matchn, sep[matchn], searchndx, RLENGTH, searchstr)
    # search for next rx
    searchndx = match(searchstr, rx)
    matchn = matchn + 1
  if (match1)  matches = split(str,fld,rx)
  return matches

  print "Test of:"
  print "  stringSplit()"
  str = "[[link|label label]][[link]] @tag more text some text with @anothertag and [[another|link]]"
  rx  = "[][][][]|@[[:alnum:]]+"
  # fld - array of fields
  # sep - array of seperators
  tags = 0
  matches = stringSplit(str,fld,rx,sep)
  # arrayDebug("fld",fld)
  # arrayDebug("sep",sep)
  # print
  print "Results:"
  printf "  "
  # per glenn jackman answer at
  #   http://unix.stackexchange.com/questions/109491/the-ere-regex-to-split-string-between-a-delimiter-and-end-of-word
  for (i=1; i<=matches; i++) {
    printf "(%s)", fld[i]
    if (match(sep[i], /^@(.+)/, m))  { printf "(%s)", m[1]; ++tags }
  print "Summary:"
  printf("  %d matches + %d tags = %d printed using regex(rx): %s\n  on string(str): %s\n", matches, tags, matches + tags, rx, str)
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.