3

I have a file with a large number of space separated columns. I want to print specific columns based on certain numerical criteria in a dynamic way. For example:

]$ cols=$(for i in `seq 1 3`; do echo -n "\$$[$[i-1]*6+1],\$$[$[i-1]*6+2],\$$[$[i-1]*6+3],\$$[$[i-1]*6+4+66],\$$[$[i-1]*6+5+66],\$$[$[i-1]*6+6+66],"; done)

which gives me the columns I want to print out:

]$ echo ${cols%?}
$1,$2,$3,$70,$71,$72,$7,$8,$9,$76,$77,$78,$13,$14,$15,$82,$83,$84

when I pass this to awk as a string, I don't get what I want:

]$ awk -v cols=${cols%?} '{print cols}' file-testawk | head -2
$1,$2,$3,$70,$71,$72,$7,$8,$9,$76,$77,$78,$13,$14,$15,$82,$83,$84
$1,$2,$3,$70,$71,$72,$7,$8,$9,$76,$77,$78,$13,$14,$15,$82,$83,$84 

awk treats it as a string rather than column identifiers.

How can I pass a string of columns to print out to awk in a way that will be correctly recognized? I'm looking for a simple, more-or-less one-liner solution, e.g. like this:

cols=$(for i in `seq 1 3`; do echo -n "\$$[$[i-1]*6+1],\$$[$[i-1]*6+2],\$$[$[i-1]*6+3],\$$[$[i-1]*6+4+66],\$$[$[i-1]*6+5+66],\$$[$[i-1]*6+6+66],"; done); awk -v cols=${cols%?} '{print cols}' file-testawk > file.out

3 Answers 3

3

awk does not have eval like capabilities but you can make a trick using awk -f capability (read script from file) combined with bash process substitution:

$ a="\$1,\$4"
$ echo "$a"
$1,$4
$ a="{print $a}"
$ echo "$a"
{print $1,$4}
$ awk -f <(echo "$a") <<<"one two three four five"
one four
1
  • Beautiful. awk -f <(echo "{print ${cols%?}}") file-testawk | head -2 works like a charm.
    – Dimi
    Jun 2, 2017 at 20:19
2

Usage: ./pass_numbers_to_awk.sh Explanation in comments.

#!/bin/bash

#generate random string of numbers - simulation column's numbers 
for i in {1..2}; do
    for j in {1..3}; do
        num=$(( (i-1) * 6 + j ))
        #numbers separated by vertical bar symbol 
        string_of_numbers+="${num}|"
    done
done

# pass to awk string like a "1|2|3|7|8|9|13|14|15|", 
# removing last vertical bar "|"
##
# use the awk split function - for information 
# look at the 'man mawk | grep -A 3 split\(s,A,r\)'
##      
# go through array and print specified columns.

awk -v string_from_bash="${string_of_numbers%?}" '
BEGIN {
    num_of_cols = split(string_from_bash, array_of_columns, "|");
}
{
    for (i = 1; i <= num_of_cols; i++) {

        # Prevent trailing spaces emergence
        OFS = (i > 1) ? " " : ""

        printf "%s%s", OFS, $array_of_columns[i];
    }
    printf "\n";
}' < input.txt

Creating the input.txt file for testing: ./create_table.sh > input.txt

#!/bin/bash

for i in {A..O}; do
    for j in {1..10}; do
        echo -n "column_${j} "
    done
    echo
done
2
  • 1
    awk for(idx in ary) is allowed to enumerate idx values in effectively random order, and almost always does, which garbles your output. gawk 4 has a nondefault option to sort them; for(i=1; i in ary; i++) ... ary[i] ... or n=split(,ary,); ... for(i=1; i<=n; i++) ...ary[i] ... work portably. Jun 4, 2017 at 7:11
  • @dave_thompson_085 Thanks, didn't know about it. Changed the code.
    – MiniMax
    Jun 4, 2017 at 18:47
1

awk is adept at doing these index calculations, so this:

awk -v N=3 '
   {
   for ( i=1; i<= N; ++i )
      print $((i-1)*6+1), $((i-1)*6+2), $((i-1)*6+3), $((i-1)*6+4+66), $((i-1)*6+5+66), $((i-1)*6+6+66)
   }
' data.file

Basic idea being, if you give awk a number stored in a variable i then awk can fetch the field corresponding to that number via $(i). Now i can as well be an expression also as is the case here.

1
  • This is good, but it is not very dynamic. Basically, I don't want to do anything in awk because for i in `seq 1 3` can change randomly, e.g. for i in 2 4 6 13 21 as well as the functional dependences on $i.
    – Dimi
    Jun 2, 2017 at 20:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .