2

Some precisions:

  • regular awk, NOT gawk
  • AIX 6.1, and old shell: GNU bash, version 2.05b.0(1)

I am trying to display some things in the right order, hence I do pipe outputs through the shell's "sort" and "uniq"... But I can't manage to get the output in the order I write it.

Troubling code:

egrep -i "something_FREQUENT" BIGFILE | sort | awk -F',' '
      { servs=servs $1 " " ; groupe=groupe "\n   " $2 ; }
  END { print "Groupes (alphabetical order):" ;
        printf groupe "\n" | "grep . | sort | uniq | xargs echo ";
        print ; rem="extra newline to help buffering... does NOT help.";
        system(""); rem="supposed to force flush of stdout... does NOT help.";
        print "Servers:"
        printf "%s\n", servs;
      } ' # if I add a final: "| cat" here (after "'"), does NOT help?... see last example

It outputs:

Groupes (alphabetical order):

Servers:
( here a quite long list of severs... maybe until "buffer size" length? )
( etc.... but at some point :)(HERE_THE_groupe_OUTPUT)(rest of the servers here....)

??... I am at a loss : it seems that if I add : system(""); before the lines to display the servers, it does NOT help either...

Some relevant tests on the same system:

1) similar to my script: "long" output through system calls, gets garbled:

prompt$:  yes "test1" | head -10 | awk '
         { all=all "\n" $0; }
     END { print "first line:"  ;
           printf all "\n" | "sort "; 
           system(""); 
           print "last line" ;
         }'

first line:
last line

test1
test1
test1
test1
test1
test1
test1
test1
test1
test1
  # notice: the order is mixed: the shell command output is last.

2) SAME but this time we add a final '| cat' : it magically reorders the output.... but this DOES NOT HELP if I do the same in my script above...

prompt$:  yes "test2" | head -10 | awk '
         { all=all "\n" $0; }
     END { print "first line:"  ;
           printf all "\n" | "sort "; 
           system(""); 
           print "last line" ;
         }' | cat  # ONLY ADDITION: pipe awk through cat... this HELPS here (but NOT in my script!... ??)

first line:

test2
test2
test2
test2
test2
test2
test2
test2
test2
test2
last line
  # notice: the order is OK!
  • ... I had to edit 10 times or almost to get the 2 examples right ^^ ... this will become community wiki before the first answer? ^^ – Olivier Dulac Sep 14 '16 at 16:05
  • Write output to two temporary files and do the sorting outside awk. No buffering, no cry. – Satō Katsura Sep 14 '16 at 17:08
  • have you tried fflush() ? or fflush("") ? – Severus Tux Sep 15 '16 at 0:45
  • @SeverusTux : good suggestion, but the one I chose (system("");) is more portable: see: gnu.org/software/gawk/manual/html_node/I_002fO-Functions.html: The fflush() function (...). However, its use is not portable to many older awk implementations. An alternative method (...) is to call system() with a null string as its argument:(...) (Other implementations may only flush the buffer associated with the standard output and not necessarily all buffered output.) (read the whole paragraph, I just edited it here to point out the relevant parts, but the meaning is a bit altered) – Olivier Dulac Sep 15 '16 at 9:19
3

You need to close the pipe inside awk to have it flushed. For your second example do:

yes "test1" | head -10 | awk '
    { all=all "\n" $0; }
END { print "first line:"  ;
      printf all "\n" | "sort "; 
      close("sort ");
      print "last line" ;
    }'

To make it more evident in your first example, do

cmd = "grep . | sort | uniq | xargs echo ";
printf groupe "\n" | cmd;
close(cmd);
  • this works, and explains why :) thanks a lot! – Olivier Dulac Sep 14 '16 at 18:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.