Can I configure my shell to print STDERR and STDOUT in different colors?

Question

I want to set my terminal up so stderr is printed in a different color than stdout; maybe red. This would make it easier to tell the two apart.

Is there a way to configure this in .bashrc? If not, is this even possible?

---

Note: This question was merged with another that asked for stderr, stdout and the user input echo to be output in 3 different colours. Answers may be addressing either question.

Answer 1

Check out stderred. It uses LD_PRELOAD to hook to libc's write() calls, colorizing all stderr output going to a terminal. (In red by default.)

Answer 2

This is a harder version of Show only stderr on screen but write both stdout and stderr to file.

The applications running in the terminal use a single channel to communicate with it; the applications have two output ports, stdout and stderr, but they're both connected to the same channel.

You can connect one of them to a different channel, add color to that channel, and merge the two channels, but this will cause two problems:

• The merged output may not be exactly in the same order as if there had been no redirection. This is because the added processing on one of the channel takes (a little) time, so the colored channel may be delayed. If any buffering is done, the disorder will be worse.
• Terminals use color changing escape sequences to determine the display color, e.g. ␛[31m means “switch to red foreground”. This means that if some output destined to stdout arrives just as some output for stderr is being displayed, the output will be miscolored. (Even worse, if there's a channel switch in the middle of an escape sequence, you'll see garbage.)

In principle, it would be possible to write a program that listens on two ptys¹, synchronously (i.e. won't accept input on one channel while it's processing output on the other channel), and immediately outputs to the terminal with appropriate color changing instructions. You'd lose the ability to run programs that interact with the terminal. I don't know of any implementation of this method.

Another possible approach would be to cause the program to output the proper color changing sequences, by hooking around all the libc functions that call the write system call in a library loaded with LD_PRELOAD. See sickill's answer for an existing implementation, or Stéphane Chazelas's answer for a mixed approach that leverages strace.

In practice, if that's applicable, I suggest redirecting stderr to stdout and piping into a pattern-based colorizer such as colortail or multitail, or special-purpose colorizers such as colorgcc or colormake.

¹ _{pseudo-terminals. Pipes wouldn't work because of buffering: the source could write to the buffer, which would break the synchronicity with the colorizer.}

Answer 3

Colouring user input is difficult because in half the cases, it is output by the terminal driver (with local echo) so in that case, no application running in that terminal may know when the user is going to type text and change the output colour accordingly. Only the pseudo-terminal driver (in the kernel) knows (the terminal emulator (like xterm) sends it some characters upon some keypress and the terminal driver may send back some characters for echo, but xterm can't know whether those are from the local echo or from what the application output to the slave side of the pseudo terminal).

And then, there's the other mode where the terminal driver is told not to echo, but the application this time outputs something. The application (like those using readline like gdb, bash...) may send that on its stdout or stderr which is going to be difficult to differentiate from something that it outputs for other things than echoing back the user input.

Then to differentiate an application's stdout from its stderr, there are several approaches.

Many of them involve redirecting the commands stdout and stderr to pipes and those pipes read by an application to colour it. There are two problems with that:

• Once stdout is no longer a terminal (like a pipe instead), many application tend to adapt their behavior to start buffering their output which means that output is going to be displayed in big chunks.
• Even if it's the same process that processes the two pipes, there's no guarantee that the order the text written by the application on stdout and stderr will be preserved, as the reading process can't know (if there's something to be read from both) whether to start reading from the "stdout" pipe or the "stderr" pipe.

Another approach is to modify the application so that it does colour its stdout and stdin. It is often not possible or realistic to do.

Then a trick (for dynamically linked applications) can be to hijack (using $LD_PRELOAD as in sickill's answer) the outputting functions called by the application to output something and include code in them that sets the foreground colour based on whether they're meant to output something on stderr or stdout. However, that means hijacking every possible function from the C library and any other library that does a write(2) syscall directly called by the application that may potentially end up writing something on stdout or stderr (printf, puts, perror...), and even then, that may modify its behavior.

Another approach could be to use PTRACE tricks as strace or gdb do to hook ourself every time the write(2) system call is called and set the output colour based on whether the write(2) is on file descriptor 1 or 2.

However, that's quite a big thing to do.

A trick which I've just been playing with is to hijack strace itself (which does the dirty work of hooking itself before every system call) using LD_PRELOAD, to tell it to change the output colour based on whether it has detected a write(2) on fd 1 or 2.

From looking at strace source code, we can see that all it outputs is done via the vfprintf function. All we need to do is to hijack that function.

The LD_PRELOAD wrapper would look like:

#define _GNU_SOURCE
#include <dlfcn.h>
#include <string.h>
#include <stdio.h>
#include <stdarg.h>
#include <unistd.h>

int vfprintf(FILE *outf, const char *fmt, va_list ap)
{
  static int (*orig_vfprintf) (FILE*, const char *, va_list) = 0;
  static int c = 0;
  va_list ap_orig;
  va_copy(ap_orig, ap);
  if (!orig_vfprintf) {
    orig_vfprintf = (int (*) (FILE*, const char *, va_list))
      dlsym (RTLD_NEXT, "vfprintf");
  }

  if (strcmp(fmt, "%ld, ") == 0) {
    int fd = va_arg(ap, long);
    switch (fd) {
    case 2:
      write(2, "\e[31m", 5);
      c = 1;
      break;
    case 1:
      write(2, "\e[32m", 5);
      c = 1;
      break;
    }
  } else if (strcmp(fmt, ") ") == 0) {
    if (c) write(2, "\e[m", 3);
    c = 0;
  }
  return orig_vfprintf(outf, fmt, ap_orig);
}

Then, we compile it with:

cc -Wall -fpic -shared -o wrap.so wrap.c -ldl

And use it as:

LD_PRELOAD=/path/to/wrap.so strace -qqf -a0 -s0 -o /dev/null \
  -e write -e status=successful -P "$(tty)" \
  env -u LD_PRELOAD some-cmd

You'll notice how if you replace some-cmd with bash, the bash prompt and what you type appears in red (stderr) while with zsh it appears in black (because zsh dups stderr onto a new fd to display its prompt and echo).

It does appear to work surprisingly well even for applications that you'd expect not (like those that do use colours).

The colouring mode is output on strace's stderr which is assumed to be the terminal. With -P "$(tty)", we're avoid doing it for writes that don't go to the terminal like when stdout/stderr have been redirected.

That solution has its limitations:

• Those inherent to strace: performance issues, you can't run other PTRACE commands like strace or gdb in it, or setuid/setgid issues
• It's colouring based on the writes on stdout/stderr of each individual process. So for instance, in sh -c 'echo error >&2', error would be green because echo outputs it on its stdout (which sh redirected to sh's stderr, but all strace sees is a write(1, "error\n", 6)).

October 2021 edit. That wrapper no longer works here on a Debian unstable with strace 5.10, glibc 2.32, gcc 10.30.0, as the function to wrap now needs to be __vfprintf_chk and the formats to look for have changed. The wrapper needs to be changed to:

#define _GNU_SOURCE
#include <dlfcn.h>
#include <string.h>
#include <stdio.h>
#include <stdarg.h>
#include <unistd.h>

int __vfprintf_chk(FILE *outf, int x, const char *fmt, va_list ap)
{
  static int (*orig_vfprintf) (FILE*, int, const char *, va_list) = 0;
  static int c = 0;
  va_list ap_orig;
  va_copy(ap_orig, ap);
  if (!orig_vfprintf) {
    orig_vfprintf = (int (*) (FILE*, int, const char *, va_list))
      dlsym (RTLD_NEXT, "__vfprintf_chk");
  }

  if (strcmp(fmt, "%d") == 0) {
    int fd = va_arg(ap, long);
    switch (fd) {
    case 2:
      write(2, "\e[31m", 5);
      c = 1;
      break;
    case 1:
      write(2, "\e[32m", 5);
      c = 1;
      break;
    }
  } else if (strcmp(fmt, "= %lu") == 0) {
    if (c) write(2, "\e[m", 3);
    c = 0;
  }
  return orig_vfprintf(outf, x, fmt, ap_orig);
}

It does show this kind of approach is a bit brittle as it uses an undocumented, unstable API (not really an API).

Answer 4

See Mike Schiraldi's Hilite which does this for one command at a time. My own gush does this for a whole session, but also has a lot of other features/idiosyncracies that you might not want.

Answer 5

Here's a proof of concept I did a while back.

It only works in zsh.

# make standard error red
rederr()
{
    while read -r line
    do
        setcolor $errorcolor
        echo "$line"
        setcolor normal
    done
}

errorcolor=red

errfifo=${TMPDIR:-/tmp}/errfifo.$$
mkfifo $errfifo
# to silence the line telling us what job number the background job is
exec 2>/dev/null
rederr <$errfifo&
errpid=$!
disown %+
exec 2>$errfifo

It also assumes you have a function called setcolor.

A simplified version:

setcolor()
{
    case "$1" in
    red)
        tput setaf 1
        ;;
    normal)
        tput sgr0
        ;;
    esac
}

Answer 6

Some previous discussion on serverfault.

See also grc and a helpful blog about it.