3

I have hundreds of logs spread across a few thousand VM's and I'm trying to create a few scripts to quickly scan logs and I have figured out the majority of the presentation and wanted to know if there is an easy way using something like awk, printf, etc. which I know how to use but specifically to print an iterated range of values from the log files?

Example:

awk -F, '{printf $(1..5)}' huge_log_file.csv
column1 column2 column3 column4 column5
etc.

I know how to properly awk, format, separate etc the printed output but thought it would be nice if I could specify a range of fields I would like it to print.

I'll update my question if I figure out how to do it with the suggested similar questions and/or on my own.

Thanks!

EDIT: I know how to print $1 $2 $3 manually but didn't include it in my example

EDIT 2: I am also aware of how to count the number of fields with awk using NF for making the range dynamic later which is my long term goal.

2

Ranges that begin with the first field

Let's consider this test file:

$ cat input.csv
a,b,c,d,e,f,g,h,i,j

With GNU awk at least, we can print the first five (or other number) of fields like this:

$ awk -F, '{NF=5; print}' OFS=, input.csv
a,b,c,d,e

For those who value conciseness over clarity, we could equivalently write:

$ awk -F, '{NF=5;} 1' OFS=, input.csv
a,b,c,d,e

Dynamic resizing

To leave off the last 2 fields, regardless of how many fields precede those two:

$ awk -F, '{NF-=2; print}' OFS=, input.csv
a,b,c,d,e,f,g,h

Other ranges

To print starting and stopping with arbitrary fields, then a loop is needed:

$ awk -v first=3 -v last=6 -F, '{for (i=first;i<=last;i++) printf "%s%s",$i,(i==last?ORS:OFS)}' OFS=, input.csv
c,d,e,f

To print from the third field and dynamically leaving off the last two fields:

$ awk -v first=3 -F, '{last=NF-2; for (i=first;i<=last;i++) printf "%s%s",$i,(i==last?ORS:OFS)}' OFS=, input.csv
c,d,e,f,g,h
1

Just a quick point; awk is a lot more flexible, but if all you want is a particular range of fields, use cut:

cut -d, -f1-5 huge_log_file.csv

This is much simpler than an awk loop if that's all you need.

  • I had considered this idea also but there are at least 4 or 5 different log formats I'll be using as the input. Using the answers from John1024 I should be able to modify my script to use these different log formats, drop the columns I don't need, convert proprietary strings into 'management friendly' text with sed, etc. – Spear Jun 14 '16 at 5:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.