I want to count the number of times that a role is successfully deleted. The problem is that one line of the log file will show that the script is about to delete the role:

Prepare to remove role X

The next line will tell me if the delete is successful or not:

Successful delete:

Delete Successful

Unsuccessful delete:

Failed to delete role X: error code

How do I count the number of successful Role deletes? I would just use the following grep:

grep "Delete Successful" | wc -l

HOWEVER, I am also deleting policies which is logged in the same manner as the roles are above. IE:

Prepare to delete policy X
Delete Successful


Failed to delete policy X: error code

Any way to search for "Prepare to delete role" on one line and then count the number of times the very next line says "delete successful?"

  • Can you not check the exit status instead? – jordanm Mar 8 '13 at 16:06

If your grep is the GNU grep, here is a quick and dirty solution:

grep -A1 "Prepare to remove role" | grep "Delete Successful" | wc -l

The grep option -A1 tells grep to print the matching line AND one line following the matching line. The second grep then only prints the lines where the delete is successfull.

Note that this will only work reliably when the "Prepare to remove role X" line is always immediately followed by the "Delete Successful" line.

Also note: you don't need wc -l because grep has that functionality built in:

grep -A1 "Prepare to remove role" | grep -c "Delete Successful"
  • how do you tell if your grep is GNU? – amphibient Mar 8 '13 at 21:06
  • 1
    @amphibient Mostly you have GNU grep if you're on non-embedded Linux or Cygwin. grep --version will tell you for sure (GNU grep will say that it's GNU grep). There are other grep implementations that support -A, such as FreeBSD's and BusyBox (if included at compile time). – Gilles Mar 8 '13 at 22:29
  • @amphibient, If it's GNU, it will also support a --version option which will tell you if it's the GNU grep. GNU grep is the default grep of most Linux distributions and most BSDs and derivatives including MacOS/X but is generally not found on other commercial Unices like Solaris, HP/UX or AIX or under a different name like ggrep or a different location like /opt/gnu/bin/grep. – Stéphane Chazelas Mar 8 '13 at 22:30
  • @Gilles, FreeBSD's grep is the GNU grep AFAIK. – Stéphane Chazelas Mar 8 '13 at 22:32
  • @StephaneChazelas Oh, right, they reimplemented tar but grabbed grep. s/Free/Open/ then, OpenBSD do keep their own implementation of grep and support -A. – Gilles Mar 8 '13 at 22:36

With awk:

awk '/Delete Successful/ && last_line ~ /Prepare to remove role/ {n++}
     END {print n+0}'

Assuming those are the only two ways "Delete Successful" can get into the log, why not just use a little bit of math?

SUCCESS=$(grep 'Delete Successful' | wc -l)
POLICY_COUNT=$(grep 'Prepare to delete policy X' | wc -l)
POLICY_FAIL=$(grep 'Failed to delete policy X' | wc -l)

(Since shell-only algorithms turned out to be vastly slower than external tools, I've simply removed any references to them. The benchmark data might be interesting though.)

Test data:

Prepare to remove role foo
Delete Successful
Prepare to remove role bar
Failed to delete role bar: 1
Prepare to remove policy baz
Delete Successful
Prepare to remove role ban
Delete Successful
Prepare to remove role bay
Failed to delete role bar: 2
Prepare to remove role bat
Failed to delete role bar: 1

Benchmark with the test data above repeated a million times (that's 306 MB), best out of three interleaved runs, sorted by increasing real time:

Test code:

for index in {1..3}
    for path in grep.sh awk.sh
        echo "$path:"
        time bash "$path"

Test system: Intel Core i7 @ 3.07 GHz with 6 GB RAM.

  • race condition: no second_line available --> stale process – H.-Dirk Schmitt Mar 8 '13 at 19:19
  • I posted some tips yesterday. – Gilles Mar 8 '13 at 22:31
  • Several things will affect the performance. The implementation of the utilities: mawk is significantly faster than gawk. The number of CPUs in the grep solution will make a difference. The locale will make a difference (older versions of gawk were known to be dead slow in multibyte locales). You may want to consider CPU time vs wall clock time. The frequency of occurrence in the input of the patterns to match may also make a difference. – Stéphane Chazelas Mar 9 '13 at 13:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.