I am able to run this and it works fine - note: field $1 is a time/date stamped field..

gawk -F ":" "{ print $1 }" /cygdrive/c/counting/ourlog | sort | uniq -c | sort -r
  57339 2014-03-21 09
  54290 2014-03-21 08
  54036 2014-03-21 10
  53254 2014-03-21 11
  52777 2014-03-21 12
  50785 2014-03-21 07
  49729 2014-03-21 16
  44459 2014-03-21 15
  43932 2014-03-21 13
  43335 2014-03-21 06
  40952 2014-03-21 14
  40864 2014-03-21 17

Now, what I want to do is to suppress the first 10 rows (they are comments preceding with a #) - and this can vary. So, we want to look for the first xx rows that begin with a #.

We change the script to:

gawk -F ":" "{ print $1 }" /cygdrive/c/counting/ourlog | sort | uniq -c | sort -r | gawk -v MyID="$id" '/#/{n++}; END {print n+0}' | gawk "NR> MyID "

but this does not work. If we make a couple more changes - we see the desired result:

gawk -F ":" "{ print $1 }" /cygdrive/c/counting/ourlog | gawk "NR>10" | sort | uniq -c | sort -r

I know I can physically type in 10 to skip the first 10 or 20 rows. However, I want that to be a calculated value.

  • 2
    How about grep -v ^# to eleminate all comments? Or do you want to supress only the "first" comments? – chaos Apr 1 '14 at 6:00
  • That may work just fine.. Let me see what I can do with that.. – Leptonator Apr 1 '14 at 6:15
  • Have a look at my sed response below... – Leptonator Apr 1 '14 at 13:44

Here is awk code to skip only the initial comments and then print $1 on the remaining lines:

gawk -F: -v c=1 '/^[^#]/ {c=0} c==0 { print $1 }' ourlog

Before the program starts, the variable c is set to 1. As soon as a non-comment line is found, c is set to zero and it stays that way for the rest of the execution. When c==0, the print statement is executed.

If you just want to eliminate all comments, then the code is much simpler:

gawk -F: '/^[^#]/ { print $1 }' ourlog

The above checks each line against the regular expression ^[^#] which matches only if the first character is not #. If it matches (line not a comment), then the print statement is executed.


Since my original Post was edited for grammatical issues.. I have to place this as a new "answer"...

Here is another way to approach this and I did not think of it until this morning..

sed "/#/d" "/cygdrive/c/!chkout/ourlog" | gawk -F ":" "{print $1}"  | sort | uniq -c | sort -r

I'm not exactly sure what you want to do because you don't show the actual input, just the output you want and various bits of code that are used in various stages. However, I think the following will do what you want (make sure you set -F: on the command line. If not, I've tried to describe each part to give you an idea of how to modify it.

!/^#/ {                                      # do the following on all rows that don't begin 
                                             # with `#`

    a[$1]++                                  # store column 1 as the key in an array and
                                             # increment the value for each occurrence

END {                                        # do the following after reading the entire file

    PROCINFO["sorted_in"] = "@ind_num_desc"; # set array traversal as numeric index descending
                                             # (requires gawk >= 4.0, otherwise, additional code
                                             # will be needed)

    for (i in a) {                           # loop through the array setting i as the index of
                                             # the current entry

        print a[i], i;                       # print the value (row count) and the index (the
                                             # row)

I don't think you need to pass a variable any more because it seems that was just used to identify how many initial comment lines to skip, but if you want to do it, you almost had it in your example, but each time you invoke gawk, it's a fresh instance. You passed the variable to the instance before the one that used it. So, in your example above, you would need to change it to:

gawk -F ":" "{ print $1 }" /cygdrive/c/counting/ourlog \
    | sort | uniq -c | sort -r                         \
    | gawk '/#/{n++}; END {print n+0}'                 \
    | gawk -v MyID="$id" "NR> MyID "

But all of that can be combined. The last two lines just count the lines with # and, I guess, tries to pass that value to another instance, but you just printed it to STDOUT, so I'm not sure how it would work. So just modify the first line to have awk skip those lines:

gawk -F: '!/^#/ { print $1 }' /cygdrive/c/counting/ourlog \
    | sort | uniq -c | sort -r

If that's what you want, and you want to avoid all the pipes, then the above code will work.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.