Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How to extract a substring of $0, in awk, when that substring's position and length is based on the positions of fields in the original $0?

It effectively equates to removing the first n.lead fields and the last n.trail fields and all those fields' leading and trailing delimiters, from the original $0

Here is an example: It is only known that the range is $4 to $8 inclusive.
The delimiter is any number of spaces and/or a single comma, eg. " , " or just ","... and the leading delimiter is to be ignored.

          Input: "   a  a  a   X marks   the   start,   Y   marks the  end  "
Expected Output: "X marks   the   start,     Y"
share|improve this question
up vote 2 down vote accepted

Awk doesn't remember the field positions or the delimiter strings. You'll have to find out the field positions manually. It's not very hard.

echo "   a  b  c   X marks   the   start,   Y   marks the  end  " |
awk '{
    i=1; n=1; tmp=$0;  # i=field number, n=column number
    while (match(tmp, / *, *| +/)) {
        A[i]=n; B[i]=n+RSTART-1;     # A[i],B[i] = start,end of delimiter i
        ++i; n+=RSTART+RLENGTH-1;
    print substr($0, A[5], B[9]-A[5])   # start at 4+1 because the first field is empty
share|improve this answer
Neat! Thanks gilles... It's good to see some awk foo,... (by the way, it needs i++)... I finally got something running myself, after much experimenting :), but it was far less flexible than your method... I got caught up in the fields, but as you've shown, there is enough info in the delimiters alone... I've posted it (because it works :) – Peter.O Jul 19 '11 at 13:48

This can be a solution, as long as there are no tabs as delimiters

#!/usr/bin/awk -f

    start = index($0, " " $4 " ")
    stop  = index($0, " " $8 " ")
    print substr($0, start+1, stop - start + length($8))
share|improve this answer
@enzotib:... Thanks, but is has a major flaw... if any of the preceding text matches $4 or $8, it produces the wrong output.. :( – Peter.O Jul 19 '11 at 6:14
So the semantic of your request is wrong, because you don't state what field "X" is the real start, and what equal to "Y" is the real end. – enzotib Jul 19 '11 at 6:43
True (I'm terrible at semantics) I intended is as a typical example only, intended for use in situations where the example line is just one of a file being input to awk.. so yes, your answer works perfectly for the example.. +1 – Peter.O Jul 19 '11 at 6:57

As it stands, it works for data with and without a leading delimiter, but will fail if any field contains "regex sensitive" data. A workaround is to substitute regex dots .{length of field} for each field... Here is a link to such a version... It is cludgey, but it won't crash because of the above mentioned issue:

awk 'BEGIN { FS = "([ \t]+)|([ \t]*,[ \t]*)" }
{ # Ignore leading delimiter, if present 
  hasLeadDlm = match($0, "^("FS")")
  LeadDlm = substr($0, 1, RLENGTH)
  if (hasLeadDlm) { sub("^("FS")", ""); }    # delete leading whitespace 
  sub("^"$1"("FS")"$2"("FS")"$3"("FS")", "") # delete leading fields
  match( $0, "^"$1"("FS")"$2"("FS")"$3"("FS")"$4"("FS")"$5)
  print substr($0, 1, RLENGTH)
}' <<< \
"a    X  a   X marks   the   start, ssY   marks the  end  
 a    X  a   X   marks the   start,  sY   marks the  end  
  a   X  a   X marks     the start,   Y   marks the  end  
   a  X  a   X marks   the     start ,Y   marks the  end  
    a   X  a   X marks   the   start,sssY   marks the  end"
share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.