Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

From this answer I have reduced a log file to this:

Timestamp:1359021601 2013-01-17 15:00:01
size: 10G   /mnt/SolrFiles/solr/api/
Timestamp:1359025201 2013-01-17 16:00:01
size: 11G   /mnt/SolrFiles/solr/api/
...snip hundreds of lines...
Timestamp:1359021601 2013-01-24 10:00:01
size: 11G   /mnt/SolrFiles/solr/api/
Timestamp:1359025201 2013-01-24 11:00:01
size: 11G   /mnt/SolrFiles/solr/api/
Timestamp:1359028801 2013-01-24 12:00:01
size: 11G   /mnt/SolrFiles/solr/api/
Timestamp:1359032401 2013-01-24 13:00:01
size: 12G   /mnt/SolrFiles/solr/api/

That pattern will carry on for hundreds of lines. I would like to reduce the file to only show the Timestamps and sizes when the size changes, like this:

Timestamp:1359021601 2013-01-17 15:00:01
size: 10G   /mnt/SolrFiles/solr/api/
Timestamp:1359025201 2013-01-17 16:00:01
size: 11G   /mnt/SolrFiles/solr/api/
Timestamp:1359032401 2013-01-24 13:00:01
size: 12G   /mnt/SolrFiles/solr/api/

Can this be accomplished using common Linux CLI tools such as grep and sed?

share|improve this question
    
What about using a few lines of Perl or Python to get this solved? –  wollud1969 Jan 24 '13 at 14:58
    
I'm open to all solutions, but ideally I would be able to add the solution as a bash alias without installing anything to $HOME/bin. Even better would be a one-liner that I could bang out from memory and impress my colleagues with! I'll be using this from a few terminals SSHed into a few different servers. –  dotancohen Jan 24 '13 at 15:01

1 Answer 1

up vote 7 down vote accepted

That's a typical job for awk:

awk '/^Timestamp/{t=$0; next}
     /^size/ && $2 != last_size {
        print t
        print
        last_size = $2
     }'

If you want to make it obscure and consise, you could do:

awk '!(/^T/&&t=$0)&&$2!=l&&(l=$2)&&$0=t RS$0'
share|improve this answer
    
Cool! I would have done it using Perl, but this is definitely better. –  wollud1969 Jan 24 '13 at 15:22
    
Thanks, Stephane! Is there any way to concise this into a single line of bash? If not then no worry. –  dotancohen Jan 24 '13 at 15:37
    
@dotancohen you can put the above awk on a single line, but leaving the newlines in won't prevent you from running it in an interactive shell. –  jordanm Jan 24 '13 at 15:40
    
Thank you, this works perfect! –  dotancohen Jan 24 '13 at 16:06
    
When I put it in an alias or shell script, I get this error: awk: line 1: syntax error at or near != awk: line 1: syntax error at or near. For instance: alias grow="cat hourly.log | sed -n '/Timestamp/{p;n;n;n;n;n;n;n;n;n;p}' | awk '/^Timestamp/{t=$0; next} ; /^size/ && $2 != last_size { print t ; print ; last_size = $2 }' | tail". On the CLI it works perfect. Any idea why? –  dotancohen Jan 24 '13 at 16:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.