To output matches in colour, grep
writes colouring escape sequences before and after the match.
Those are instructions to the terminals to change their background and/or foreground colour.
It's important to realise that it is in the output along with the text. You don't see it because your terminal doesn't display them as graphical symbols but instead understand it as special instructions.
The escape sequences start with an ESC character (0x1b byte (033 in octal) in ASCII aka \e
or ^[
) and are followed by a few characters which themselves don't have to be control characters.
You can reveal those characters by piping the output to things like:
$ echo + | grep --color=always . | sed -n l
\033[01;31m\033[K+\033[m\033[K$
$ echo + | grep --color=always . | od -An -vtc -tx1 -to1
033 [ 0 1 ; 3 1 m 033 [ K + 033 [ m 033
1b 5b 30 31 3b 33 31 6d 1b 5b 4b 2b 1b 5b 6d 1b
033 133 060 061 073 063 061 155 033 133 113 053 033 133 155 033
[ K \n
5b 4b 0a
133 113 012
(here also including hexadecimal and octal values of the individual bytes)
Or (though non-standard and ambiguous):
$ echo + | grep --color=always . | cat -A
^[[01;31m^[[K+^[[m^[[K$
You can see that in the grep
output, there's a \e[01;31\e[K
before the match and \e[m\e[K
after the match.
What escape sequences a given terminal recognises and how varies with the terminal. For xterm for instance, see the specification there. Those above these days are rather ubiquitous.
For the one that starts with \e[
and ends in m
, the terminal understands each of the ;
-separated numbers as different rendering attributes to apply to text that is going to be written from now on. For instance 1
is for bold, 31
sets the foreground colour to red.
\e[K
is the escape sequence that tells the terminal to clear the screen from the cursor to the end of the line.
So the terminal actually sees:
<bold-fg_red><clear-to-eol>+<reset-all-attributes><clear-to-eol>
But all tr
sees is those ESC, [
, ... m
along with the other ones that it's been asked to transliterate.
In particular here, it will transliterate m
to M
, and the escape sequence that was changing colour attributes will turn into something else together.
To find out about escape sequences and what they do, other than looking at the terminal documentation (such as https://www.invisible-island.net/xterm/ctlseqs/ctlseqs.html mentioned above for xterm), which is sometimes hard to find or inexistent, you can also look in the terminfo
database which records the escape sequences recognised by a number of terminals for a few common actions.
You can query that database by hand for your terminal (identified by the $TERM
environment variable) with infocmp
:
$ infocmp -xL1 | grep M,
delete_line=\E[M,
key_enter=\EOM,
key_mouse=\E[M,
parm_delete_line=\E[%p1%dM,
scroll_reverse=\EM,
And for the details of what those actions (capabilities) are, you can look at the terminfo(5)
man page (man 5 terminfo
).
\e[M
(delete_line
) deletes one line, \e[<decimal>M
(parm_delete_line
) deletes <decimal>
lines. So your colouring sequences have turned into line deleting sequences once transliterated to upper case.
You generally don't want to post-process coloured output as those are only intended for terminals. That's why most commands that support colouring disable it when their output doesn't go to a terminal.
For GNU grep
, as you already found out, you need --color=auto
(or grep --color
) to get that behaviour.
Now, if you do still want to see colours, you need to move the colouring to the last command in the pipeline, the one that has its output go to the terminal:
<file tr '[:lower:]' '[:upper:]' | grep -xE --colour=auto '.{5}'
Here using --colour=auto
so that if ever the script that contains that command has its output redirected / post-processed, the colouring is disabled.
Here, since the regexp matches the whole line (-x
option above, which avoids having to use ^
and $
like in your approach), you might as well switch the foreground colour to red before and clear attributes after:
if [ -t 1 ]; then
tput setaf 1 # set ANSI foreground colour
tput bold
fi
grep...
if [ -t 1 ]; then
tput sgr0 # turn off all attributes
fi
Here using tput
to query terminfo
for the right sequence for your terminal, though since most terminals do it the same, do like grep
and hardcode the sequences:
[ -t 1 ] && printf '\33[1;31m'
grep...
[ -t 1 ] && printf '\e[m'
Using [ -t 1 ]
to check that stdout (file descriptor 1) is a terminal.