3

I try, with no success, to use an awk command inside a for loop.

I've got a variable which contains a series of strings that I want to cut with awk to get the data.

I know how to do that but what I really want is to cut the data successively.

So I've got this variable:

var="data1,data2,data3"

And here where I am right now:

for ((i=1; i<=3; i++))
do
    echo $(awk -F, '{print $1}' <<< $var)
done

I try to replace the $1 by the loop $i but without success.

4
  • Every character in between a pair of single quotes is treated as a literal character.
    – Niko Gambt
    Jan 27, 2019 at 11:50
  • 1
    What is your real use-case? looping over awk seems unnecessary here (for exaple in bash you could use a parameter substitution echo "${var//,/$'\n'}") Jan 27, 2019 at 12:04
  • Actually the variable will contains urls from a zenity form. I want to use these urls separately so I have to get each one of them independently.
    – Amargein
    Jan 27, 2019 at 19:05
  • Relevant, but not a good solution in this case: How to assign value at run time in AWK command
    – Kusalananda
    Jan 27, 2019 at 22:54

4 Answers 4

6

You can accomplish what you're trying to do by using double quotes in the awk script to inject the shell variable into it. You still want to keep one literal $ in it, which you can do by escaping it with backslash:

echo $(awk -F, "{print \$$i}" <<<$var)

This will expand the $i to 1, 2 and 3 in each of the iterations, therefore awk will see $1, $2 and $3 which will make it expand each of the fields.

Another possibility is to inject the shell variable as an awk variable using the -v flag:

echo $(awk -F, -v i="$i" '{print $i}' <<<$var)

That assigns the awk variable i to the contents of the shell variable with the same name. Variables in awk don't use a $, which is used for fields, so $i is enough to refer to the i-th field if i is a variable in awk.

Assigning an awk variable with -v is generally a safer approach, particularly when it can contain arbitrary sequences of characters, in that case there's less risk that the contents will be executed as awk code against your intentions. But since in your case the variable holds a single integer, that's less of a concern.

Yet another option is to use a for loop in awk itself. See awk documentation (or search this site) for more details on how to do that.

6
  • 1
    ... or set the input record separator appropriately awk -v RS='[,\n]' 1 <<< "$var" (or maybe more portably printf "$var" | awk -v RS=, 1 ) Jan 27, 2019 at 12:06
  • @steeldriver That awk script will print all three fields at once. Which is ok given the OP is doing just that... I ended up focusing the answer on extracting a single field on the assumption they were interested in executing other commands in the shell loop (which might have been the motivation for using one in the first place...)
    – filbranden
    Jan 27, 2019 at 12:13
  • 1
    Understood - that's why I commented on the OP to clarify what they really want to do ;) Jan 27, 2019 at 12:15
  • 1
    The first is an injection (and a security concern), the second one (using -v) is not an injection.
    – Kusalananda
    Jan 27, 2019 at 12:41
  • 1
    Thanks, your answer solved my problem! First time I post a question here after years of reading posts. Not disappointed. Such a great community!
    – Amargein
    Jan 27, 2019 at 19:06
1

Using awk seems excessive in this circumstance, how about a tr and a while-loop:

tr , '\n' <<<"$var" | while read; do
  echo $REPLY
done

Output:

data1
data2
data3
2
  • Great. REPLY being the default name argument for read built-in shell command.
    – user86041
    Jan 27, 2019 at 12:21
  • 2
    Note that this requires that the shell uses a default variable to put the data in from read, which bash happens to do, but this is not standard. Also, your read command may modify the data if it contains backslashes, and may strip flanking whitespace from the values. It would also read values with embedded newlines as multiple values. You additionally need "$REPLY" to stop the shell from splitting the value and from performing filename expansion on it.
    – Kusalananda
    Jan 27, 2019 at 12:40
1
#!/bin/sh

var='data1,data2,data3'

unset data
while [ "$var" != "$data" ]; do
    data=${var%%,*}    # delete first comma and the bit after it
    var=${var#*,}      # delete bit up to first comma (and the comma)

    printf 'data = "%s"\n' "$data"
done

Here, we use variable substitutions to get each successive comma-delimited data field from the value of the var variable. The first assignment to data in the loop will remove everything from $var after the first comma. The var variable is then modified so that the first bit up to the first comma is deleted.

This continues until "$var" = "$data" which means that nothing more can be done to the string.

This way of doing it would allow us to handle comma-separated data strings that contain embedded newlines:

var='line1
line2,data2,last bit
goes here'

With the above values in var, the above script would output

data = "line1
line2"
data = "data2"
data = "last bit
goes here"

Not caring about embedded newlines; You very seldom have to loop over invocations of awk.

Note that awk is perfectly happy to read your string as a set of comma-delimited fields, and that it's able to loop over these:

printf '%s\n' "$var" |
awk -F ',' '{ for (i=1; i<=NF; i++) print $i }'

With var='data1,data2,data3', this would print

data1
data2
data3

Another shell solution that makes use of the IFS variable to split the $var value into bits while also using set -f to disable filename expansion:

set -f
oldIFS=$IFS; IFS=','

set -- $var

IFS=$oldIFS; unset oldIFS
set +f

for data do
    printf 'data = "%s"\n' "$data"
done
1

can accept both j (as variable) and $j (as field index):

for i in 1 2 3; do echo "$var" | awk -v j=$i -F , '{print $j}'; done

$i in the example "confused" awk which one to use (shell or its own variable - taking precedence) as both are referred to with $ prefix.

note

shell which is standard for "portable" scripting do not support:

(( i=1; i<=3; i++; )) and <<< $var constructs

Also you might consider using seq command in for loop for finer control in number sequence generation, if available.

1
  • 3
    Your loop would only work if you happened to have three files called 1, 2 and 3 in the current directory. Also, you use variable expansion unquoted in the shell which may have unwanted consequences if the data contains filename patterns (like *). The echo may furthermore modify the data if it contains backslashes.
    – Kusalananda
    Jan 27, 2019 at 13:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .