Awk command inside a for loop

Question

I try, with no success, to use an awk command inside a for loop.

I've got a variable which contains a series of strings that I want to cut with awk to get the data.

I know how to do that but what I really want is to cut the data successively.

So I've got this variable:

var="data1,data2,data3"

And here where I am right now:

for ((i=1; i<=3; i++))
do
    echo $(awk -F, '{print $1}' <<< $var)
done

I try to replace the $1 by the loop $i but without success.

Answer 1

You can accomplish what you're trying to do by using double quotes in the awk script to inject the shell variable into it. You still want to keep one literal $ in it, which you can do by escaping it with backslash:

echo $(awk -F, "{print \$$i}" <<<$var)

This will expand the $i to 1, 2 and 3 in each of the iterations, therefore awk will see $1, $2 and $3 which will make it expand each of the fields.

Another possibility is to inject the shell variable as an awk variable using the -v flag:

echo $(awk -F, -v i="$i" '{print $i}' <<<$var)

That assigns the awk variable i to the contents of the shell variable with the same name. Variables in awk don't use a $, which is used for fields, so $i is enough to refer to the i-th field if i is a variable in awk.

Assigning an awk variable with -v is generally a safer approach, particularly when it can contain arbitrary sequences of characters, in that case there's less risk that the contents will be executed as awk code against your intentions. But since in your case the variable holds a single integer, that's less of a concern.

Yet another option is to use a for loop in awk itself. See awk documentation (or search this site) for more details on how to do that.

Answer 2

Using awk seems excessive in this circumstance, how about a tr and a while-loop:

tr , '\n' <<<"$var" | while read; do
  echo $REPLY
done

Output:

data1
data2
data3

Answer 3

#!/bin/sh

var='data1,data2,data3'

unset data
while [ "$var" != "$data" ]; do
    data=${var%%,*}    # delete first comma and the bit after it
    var=${var#*,}      # delete bit up to first comma (and the comma)

    printf 'data = "%s"\n' "$data"
done

Here, we use variable substitutions to get each successive comma-delimited data field from the value of the var variable. The first assignment to data in the loop will remove everything from $var after the first comma. The var variable is then modified so that the first bit up to the first comma is deleted.

This continues until "$var" = "$data" which means that nothing more can be done to the string.

This way of doing it would allow us to handle comma-separated data strings that contain embedded newlines:

var='line1
line2,data2,last bit
goes here'

With the above values in var, the above script would output

data = "line1
line2"
data = "data2"
data = "last bit
goes here"

---

Not caring about embedded newlines; You very seldom have to loop over invocations of awk.

Note that awk is perfectly happy to read your string as a set of comma-delimited fields, and that it's able to loop over these:

printf '%s\n' "$var" |
awk -F ',' '{ for (i=1; i<=NF; i++) print $i }'

With var='data1,data2,data3', this would print

data1
data2
data3

---

Another shell solution that makes use of the IFS variable to split the $var value into bits while also using set -f to disable filename expansion:

set -f
oldIFS=$IFS; IFS=','

set -- $var

IFS=$oldIFS; unset oldIFS
set +f

for data do
    printf 'data = "%s"\n' "$data"
done

Answer 4

awk can accept both j (as variable) and $j (as field index):

for i in 1 2 3; do echo "$var" | awk -v j=$i -F , '{print $j}'; done

$i in the example "confused" awk which one to use (shell or its own variable - taking precedence) as both are referred to with $ prefix.

note

sh shell which is standard for "portable" scripting do not support:

(( i=1; i<=3; i++; )) and <<< $var constructs

Also you might consider using seq command in for loop for finer control in number sequence generation, if available.