How to expand variable within a single-quoted argument?

Question

I am trying to perform the following without luck:

SORT_BY='-k3,3r -k2,2 -k1,1r'
awk 'NR<4{print $0;next}{print $0 | sort '"${SORT_BY}"' -t"~"}'

I have tried with all sorts of quotes, unquote, etc.. but there is always a problem on awk

awk: illegal statement

How should I do the expansion>

Answer 1

awk isn't shell. You can't call UNIX tools like sort directly from awk any more than you could directly call them from a C program.

This would make what you're trying to do function:

SORT_BY='-k3,3r -k2,2 -k1,1r'
awk 'NR<4{print; next} {print | "sort -t\"~\" '"${SORT_BY}"'"}'

but it's not at all clear that that's a good idea.

Guessing that what you're trying to do is print 3 header lines unsorted and then the rest of the data sorted, this might be the best approach since, other than setting awks output separator and sorts input separator to the same character, it completely decouples awk from the sorting, doesn't require awk to spawn a shell, and will work whether the input is coming from a pipe or a file since it doesn't require opening the input twice:

sep='~'
awk -v OFS="$sep" '{print (NR>3), NR, $0}' file |
sort -t "$sep" -k1,1n -k2,2n -k5,5r -k4,4 -k3,3r |
cut -d "$sep" -f3-

Answer 2

Variables are not expanded in single quoted strings. There's no way around this.

In this case, you want to output the three first lines of your data as is, and sort the rest. For simplicity, I would do this with

sort_opts=( -k3,3r -k2,2 -k1,1r )

{  head -n 3 file
   awk 'NR > 3' file | sort "${sort_opts[@]}" -t '~'  } >newfile

If you feel that you really want to do the sorting from within awk, and assuming that $IFS remains unchanged with a space as its first character:

sort_opts=( -k3,3r -k2,2 -k1,1r )

awk -v sort_opts="${sort_opts[*]}" '
    BEGIN { sortcmd = sprintf("sort %s -t \"~\"", sort_opts) }
    NR <= 3 { print; next } { print | sortcmd }' file >newfile

This gives the sorting options to awk using -v on the command line, the BEGIN block creates the sorting command to use as a string in the sortcmd variable, and this is then used in the last block (triggered for lines 4 on-wards).

Note that when using print with a pipe in awk, the right hand side of the pipe must be a string representing the shell command to pipe to. This is not the case in your code, which is why you get an "illegal statement" error.

---

When values that you need to import into awk contains backslashes, you may want to use an environment variable instead (to avoid having to double up each backslash):

sort_opts=( -k3,3r -k2,2 -k1,1r )

SORT_OPTS="$sort_opts[*]" awk '
    BEGIN { sortcmd = sprintf("sort %s -t \"~\"", ENVIRON["SORT_OPTS"]) }
    NR <= 3 { print; next } { print | sortcmd }' file >newfile