I've been assisting with some reporting at work, and am trying to get a unique count of different elements from some rather large log files that we are generating. So far, I've been able to run two separate commands to come up with the counts, but due to some restrictions, I'm going to have to combine them. Here is what I have been running (the names of files, directories, search items have been changed to protect the innocent):
Command 1 - filter by _transformer_
and write all these unique log entries to a new file (each item that I'm counting has multiple entries per transaction, so I do this to speed up the next step as well as remove duplicates):
zgrep -a _transformer_ /files/are/located/here/logfile_date.gz > \
/temp/directory/count_file_date.gz
Command 2 - count the number of instances of each of several items related to the transactions:
zgrep -caE 'shockwave|starscream|megatron|prowl|blaster' \
/temp/directory/count_file_date.gz
This has worked great, but I want to combine them into one command and skip writing the new file. This is what I thought would work, but isn't:
Single Command
zgrep -a _transformer_ | \
zgrep -acE 'shockwave|starscream|megatron|prowl|blaster' \
/files/are/located/here/logfile_date.gz
Running the above command outputs a count of all log entries with the words between the pipes, and doesn't only count the ones with transformer in the particular line of the log.
zgrep
and GNUgrep
:zgrep -a _transformer_ /files/are/located/here/logfile_date.gz | grep -acE 'shockwave|starscream|megatron|prowl|blaster'