6

I have a small AWK script that summarizes the output of another program, and I am trying to print that summary before the command itself, but I cannot figure out a good way to do it. My naive first solution was running the command twice, once to get the summary info and once to print the full output with the prepended summary. Now I am using tee to only read the program output once and using head and tail to move it around, which is about 3 times faster than the previous version.

The current version looks like

program | awk ... | tee >(head -n -3) >(tail -n -3) > /dev/null

which does give the desired output but is less elegant (and fast) than I'm sure is possible. At the very least I think there is a way not to have to redirect to /dev/null, but I can't figure out the arrangement that makes that happen.

I have seen some answers on similar questions using ed, and I would like to do this as well, but when I try piping my commands into ed it doesn't work. Specifically, I thought

echo -e "$-2,$m0\n,p\nQ" | ed <(program | awk ...)

should work since running those commands in ed itself does, but I have narrowed down the problem to the "$-2" address, which appears not to work when piping into ed but does work when running it directly. I've tested just that part on both GNU ed 1.9 and 1.16. Using the -v option in both cases just gives me the "Invalid command suffix" error. Even running

echo "$-2,$p" | ed -v filename

produces this error, so it's not a problem with the process substitution. I obviously doubt I've identified a bug in ed after using it for less than an hour, so I figured I would ask here to see what I'm doing wrong. I am also open to any solutions not involving ed, I just thought it seemed like a promising way to go.

If it helps, "program" is qstat and the AWK is just counting the number of jobs running, queued, and exiting for my user and printing the totals at the end. There are probably some qstat options that would do what I want built in, but I haven't found them in the manual yet so that would be a helpful answer as well! I would really like to understand what is going wrong with my ed command though.

  • Using ed for anything is pretty archaic. You're already using awk for part of this so why not just use awk for all of it? If you post a new question with a MCVE then we could help you. – Ed Morton Jul 12 at 1:02
  • @EdMorton I would love to just use awk for this, but I don't know how yet. I have added my current awk code and example input and output. I hope that is helpful. I've been inspired by some of your previous awk answers so I appreciate any insight you have. – Brent Jul 12 at 2:34
  • 1
    Sorry for the misunderstanding. I think I'll figure it out on my own. – Brent Jul 12 at 13:25
7

You have actually stumbled on the shell, not on ed. This

echo -e "$-2,$m0\n,p\nQ"

means $- and $m0 undergo parameter expansion, as they are enclosed by double quotes. Run echo "$-" and echo "$m0" to see it for yourself. They should be enclosed in single quotes so that the shell does not expand them.

Since we are fixing it, let us also favor printf over echo. The latter has a non-uniform behavior accross implementations, while the former is sound. This should do:

printf '%s\n' '$-2,$m0' ',p' 'Q' | ed -s <(program | awk ...)

-s option has been added to ed, so as to "suppress diagnostics, byte counts and '!' prompt". This is purely cosmetic.

Sample execution (with a useless use of cat to simulate the process substitution):

$ cat input
ATOM    126  CD  GLN A 449      -2.853  11.592 119.709  1.00 17.95           C
ATOM    127  OE1 GLN A 449      -4.056  11.297 119.695  1.00 20.83           O
ATOM    128  NE2 GLN A 449      -1.948  10.876 120.359  1.00 14.98           N
HETATM  129  N   MSE A 450      -4.523  16.830 119.280  1.00 14.88           N
HETATM  130  CA  MSE A 450      -5.537  17.804 118.911  1.00 15.65           C
$ printf '%s\n' '$-2,$m0' ',p' 'Q' | ed -s <(cat input)
ATOM    128  NE2 GLN A 449      -1.948  10.876 120.359  1.00 14.98           N
HETATM  129  N   MSE A 450      -4.523  16.830 119.280  1.00 14.88           N
HETATM  130  CA  MSE A 450      -5.537  17.804 118.911  1.00 15.65           C
ATOM    126  CD  GLN A 449      -2.853  11.592 119.709  1.00 17.95           C
ATOM    127  OE1 GLN A 449      -4.056  11.297 119.695  1.00 20.83           O
| improve this answer | |
  • 2
    Good catch with parameter expansion ! +1 – Sergiy Kolodyazhnyy Jul 11 at 19:57
  • Ah, of course! I was trying this locally in fish and it gave me an error about the "$-" not being a valid variable, but I just assumed this was a fish-specific problem since when I switched to bash it went away. Thank you also for explaining why the previous ed answers I had seen used printf instead of echo because I was wondering that too! As a bit of a follow-up is it better to use the single quotes to prevent the expansion or to escape the $? I didn't realize the difference between single and double quotes either, so maybe the escaping it is slightly different too. – Brent Jul 12 at 2:09
  • 1
    @Brent Glad to help. I find single quotes resulting in a more readable command than escaping the $, and with single quotes you can be sure nothing inside will be changed by the shell. But it is up to you. (I also took the liberty to rollback your edit, as Ed Morton has explained.) – Quasímodo Jul 12 at 11:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.