4

I would like to change the way I display a big (~6000 lines) log file using wim, less or whatever, in order to simplify the problem checking.

I'd like to highlight some line of the log based on a patter (i.e. error, warning, info...) and/or hide some others.

Which tools could I use? Do I just need a shell script? It's important that after the process, I can read the output using less, vim, ... to perform search operations!

Edit: a little piece of the log:

2016/10/25 12:19:24.403355 INFO <ServiceManager.cpp#2614 TID#3> Security object has NOT been parsed
2016/10/25 12:19:24.403369 INFO <ServiceManager.cpp#1263 TID#3> Service object sequence started
2016/10/25 12:19:24.403372 DBG <ServiceManager.cpp#1276 TID#3> preinvoke succeeded
6

I would recommend a shell script, based on awk like Valentin B. solution:

$ cat colorize
awk '
function color(c,s) {
        printf("\033[%dm%s\033[0m\n",30+c,s)
}
/error/ {color(1,$0);next}
/success/ {color(2,$0);next}
/warning/ {color(3,$0);next}
/INFO/ {color(4,$0);next}
/DBG/ {color(5,$0);next}
{print}
' $1

In order to be able to interactively view the colorized output, I would use less in raw mode, e.g.:

colorize mylog.txt | less -R
  • Yes defining a function in your awk script seems appropriate to avoid being redundant. Also thanks for the raw mode in less, that is great ! Works with my solution too by piping the output. – Valentin B. Oct 25 '16 at 12:26
  • I didn't know that awk allows to use function like this!!! Thank you – Peppe Oct 26 '16 at 7:15
  • EDIT: See also: stackoverflow.com/questions/981601/… What about sed command? something like .. sed /INFO/s/^/<begin_color>/ and something with /$/<end_color>. Where begin would be \033[...` and end will be \033[0m. – danger89 Mar 20 '19 at 16:54
3

awk, as mentioned in other answers, is definitely the tool to reach for first.

But it isn't the only tool, by a long shot.

A. L. Lambert's logtool is specifically designed for post-processing log files, and has a complex (and alas badly documented) configuration system that permits one to assign a file full of regular expressions to each one of 13 colours.

cat *.log | logtool

It has the distinction of understanding logs that have TAI64N timestamps.

To that, add:

problems with colourization

Alas, a few of these tools do colourization very wrongly. They hardwire control sequences, rather than using setaf/setab and so forth from terminfo.

Also note that colourization is tricky for subtler reasons, and almost no colourization program gets it right. To get it completely right, a colourizer has to deal with automatic margins and the DEC VT pending line wrap mechanism, which I have yet to see any colourizer do. GNU grep has a fairly famous colourization bug in this area, but this is not a problem that is limited to grep.

Further reading

1

After checking, you can directly output colored text in the terminal using awk. With the example you provided you can create a awk script file (e.g. displayLog.awk) containing the following code:

# output INFO lines in cyan
$3 == "INFO" {
    print "\033[36m"$0"\033[0m"
    next
}

# don't display DBG lines
$3 == "DBG" {
    next
}

# output WARNING lines in bright yellow
$3 == "WARNING" {
    print "\033[1;33m"$0"\033[0m"
    next
}

# output ERROR lines in bright red
$3 == "ERROR" {
    print "\033[1;31m"$0"\033[0m"
    next
}

# If you want to skip all other lines, comment next line
{print}

Then open a clean terminal window, check the display is unlimited (you can display all 6000 lines at once) and use it this way :

$ awk -f displayLog.awk log.txt

OR Like user jlliagre suggested in his solution, pipe it to raw mode less:

$ awk -f displayLog.awk log.txt | less -R

That should do the trick ! You can fiddle with the colors, and what lines you want to display based on the sample awk code in my answer. More on color coding here.

EDIT

If you want only one word to be colored instead of a whole line (for example you want ERROR to be in red only) do this:

$3 == "ERROR" {
    $3 = "\033[1;31m"$3"\033[0m"
    print
    next
}
-1

If you want to to highlight some line of the log based on a pattern (i.e. error, warning, info...) then please use the below command :

grep -rn "pattern" <logfile>

This command will display all the complete lines of the logfile which will have the pattern as mentioned above.

  • OP wants colored output, not just to dump line numbers, along with the matched line. Also -r is absolutely redundant when working on a single file. – heemayl Oct 25 '16 at 10:54

Not the answer you're looking for? Browse other questions tagged or ask your own question.