1

I would like to sort file in parts delimeted by string, but it is a larger part of processing in the awk.

I have read that it could be possible to store lines between delimeter and then use asort for sorting, but I need to sort it using the second column.

I have decided to store output between delimeter in a variable (k), then pass the value of variable into command using pipe |, and, at the end read the output, process it and print.

However, using the print variable|"sort -k2,2"|getline v makes the getline(?) to stuck.

I can execute command in uglier way using system() function, but then I think that I need to use echo to pass variable to command, and the output is not stored in variable v, but the error code.

$ awk 'BEGIN{ cmd="sort -k2,2"; k="1\tB\n2\tA"; v=system("echo \""k"\" | "cmd); print "OK: "v}'
2       A
1       B
OK: 0

The nicer way with print k|command (removing |getline v) executes command, but the print "OK" is done before output from subcommand. The output is also not stored in variable v.

$ awk 'BEGIN{ cmd="sort -k2,2"; k="1\tB\n2\tA"; print k|cmd; print "OK: "v}'
OK:
2       A
1       B

How can I store command output in variable or at least to wait for the previous subcommand to finish before continuing?

3
  • what specifc thing you want to do that you use awk? Cannot you use just sort -k2,2 inputfile directly? because nothing you validate for the command result or anything in awk. what is your input and expected output? can you edit and show these instead? Feb 17, 2021 at 12:31
  • 1
    ... the piece you're missing is probably a close(cmd) statement to prevent sort from waiting for more input. See for example close() function Feb 17, 2021 at 12:46
  • @αғsнιη the awk is processing output from the other command (namely squeue), and I don't need full information about the other users' jobs, and I want to collapse them, but I still need to leave my submitted jobs in right place to know, when they will be eligible for the run.
    – faramir
    Feb 17, 2021 at 20:55

2 Answers 2

4

Is this what you're trying to do (using GNU awk for coprocesses)?

$ cat tst.awk
BEGIN {
    cmd = "sort -k2,2"
    k = "1\tB\n2\tA"

    print k |& cmd
    close(cmd, "to")

    while ((cmd |& getline line) > 0) {
        v = v line ORS
    }
    close(cmd)

    print "OK:"
    printf "%s", v
}

$ awk -f tst.awk
OK:
2       A
1       B

Since the above uses GNU awk (and you already mentioned asort() in your question which is gawk-only) and gawk already has sorting built in, you probably don't need to do that anyway, e.g. using sorted_in instead of creating a subshell for every call to sort as the above would do:

$ cat tst.awk
BEGIN {
    k = "1\tB\n2\tA"

    split(k,lines,/\n/)
    for (i in lines) {
        lines[i] = gensub(/[^\t]*\t([^\t]+)/,"\\1\t&",1,lines[i])
    }

    PROCINFO["sorted_in"] = "@val_str_asc"
    for (i in lines) {
        v = v gensub(/[^\t]\t/,"",1,lines[i]) ORS
    }

    print "OK:"
    printf "%s", v
}

$ awk -f tst.awk
OK:
2       A
1       B

but I assume this is a simplified version of some more difficult task you're trying to perform that actually would require coprocesses.

0
0

Anyone interested in the solution of the problem described in the comment:

the awk is processing output from the other command (namely squeue), and I don't need full information about the other users' jobs, and I want to collapse them, but I still need to leave my submitted jobs in right place to know, when they will be eligible for the run.

alias squeue="_squeue() {
    /usr/bin/squeue --format=\"%.8i %.9P %.20j %.10u %.2t %.10M %.10L %.6D %R\" \$@ \
    | awk '
        BEGIN { cmd=\"sort -k2,2 -k5,5 -k4,4\" }
        function sort(buf) {
            sorted=\"\"
            gsub(\"^\"ORS\"*|\"ORS\"{2,}|\"ORS\"*$\", \"\", buf)
            if (length(buf)>0) {
                print buf |& cmd
                close(cmd, \"to\")
                while ((cmd |& getline line) > 0) {
                    sorted = sorted line ORS
                }
                close(cmd)
            }
            return sorted
          }
          NR <= 1 { print; next }
          /'\$USER'/ {
              printf(\"%s\", sort(buf))
              buf=\"\"
              print
              partition=\$2; user=\$4; st=\$5;
              next
          }
          { buf = buf ORS \$0 }
          END {
              printf(\"%s\", sort(buf))
          }
        ' \
    | awk '
        BEGIN { partition=\"\"; user=\"\"; st=\"\"; c=0; n=0; }
        function print_count() {
            if(c>0) { print \" (count: \"c\" jobs, \"n\" nodes)\"; c=0; n=0; }
        }
        NR <= 1 { print \$0; next }
        /'\$USER'/ {
            print_count()
            print;
            next
        }
        \$2!=partition || \$4!=user || \$5!=st {
            print_count()
            printf(\"%-98s\", substr(\$0,0, 98));
            partition=\$2; user=\$4; st=\$5;
        }
        { n+=\$8; c++ }
        END {
            print_count()
        }' \
    |  sed '/(count: .*)$/{s/.*/\x1b[90m&\x1b[0m/p;d};
            /\s'\$USER'\s\+[^R]\+\s/{s/.*/\x1b[1m&\x1b[0m/p;d}'
}
_squeue"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.