the output of “column” is misaligned when the text is colored [duplicate]

Question

I have a list of sentences in a tcsh script, which are separated by '\n' and may have white spaces inside. I want to show them in the style of ls, so I did "echo $list| column". it works fine when there is no color used. the result is as below

Then I need to highlight some items which are changed(latest 3 choices) each time to show up. I add the color sequences (\e[33m and \e[0m) around the items I want to highlight, and still put them into the $list. When I show them with "echo $list | column", the color is ok, but the text are misaligned as following picture.

Note, I tried that if not use '| column', the strings showed correctly in one column, even though not the style I want.

Is there is some solution for the problem?

BTW, I did some search and it seems to be a bug of column. Is there any maintainer for column? Where should I report the issue?

NOTE: I've read Issue with column command and color escape codes before the post. His format is fixed, while mine to be colored items are dynamic. So simply use printf or add color after column can not work

Answer 1

The escape sequences that colorize output are not understood by column, which operates by merely counting “printable characters” in its input. As an example, the sequence to turn text green (^[[0;32m) contains six printable characters, and will count as such.

If you want to see what column really sees when it determines column width, try something like this:

ls --color=always / | tr -dc '\n -~' | column -s '\n'

Replace ls --color=always / with any command that has colored output if your ls does not support this option.

There is no way to convince column to ignore these escape sequences, but you can instead add the color afterward.

Say you have a file to-list that looks like this:

1: avifsc01
2: avifsh01_bg4dtva0f
3: avifsh01_bg5dtvz1f
4: bg2ctqz1
5: bg2dtva0
6: bg3cda0
7: bg3cda1
8: bg3cdpa0
9: bg3z1
10: bg4cdpa0f
11: bg4cdpz1f
12: bg4cdz1f
13: bg4cta0f
14: bg4ctpa0f
15: bg4ctpz1f
16: bg4ctz1f
17: bg4dtva0f
18: bg4dtvz1f
19: bg5ctpz1f
20: bg5ctz1f
21: bg5dtvz1f
22: bg6ctz1f
23: bgpk01
24: dsc
25: test
26: vpp01
27: vrdlib01

Then you do:

list=$(cat to-list)

And suppose you know you want to color a few specific entries:

to_color='bg3cda0 bg4dtva0f bg6ctz1f'

You can format the list with column before colorizing it:

printf '%s\n' "${list}"                         \
| column -s '\n'                                \
| sed "s@$( (printf '%s\\|' ${to_color}; echo)  \
            | sed 's@^@\\([0-9][0-9]*: \\(@'    \
            | sed 's@\\|$@\\)\\)@'              \
       )@$(printf '\033')[33m\1$(printf '\033')[0m@g"

These entries can be dynamically generated so long as you have some way to generate these two variables (${list} and ${to_color}).

The result:

Answer 2

The colorizing sequences are messing up the character count for column so you can add color after the columnizing step , as in:

set items2color = ( "bg3cda0" "bg4dtva0f" "bg6ctz1f" )
echo "$list" | column |\
perl -slpe '\
   for my $c ( split /\s+/, $C ) {\
      s/(?:(?<=^)|(?<=\t))\d+:\s+\Q$c\E(?=\t|$)/\e[31m$&\e[0m/;\
   }\
' -- -C="$items2color"

The particular items that you wanted colorized, are placed on the command line in the option -C, you may change any name though, which then is available to Perl via the $C variable. Since it is a space-separated list of items, we split it up on spaces and apply the regex on the current line. The backslashes at the end of lines are for the tcsh since you said it's a tcsh list variable you were operating on.

Regex

A brief working of the regex is in order and given below: Note that since the regex is not attached to any variable, it means it attaches to the $_ variable, which is the current line in our case.

/
   (?:             # From where we are,
      (?<=^)       # we see the beginning of the current line to our left
         |         # OR
      (?<=\t)      # we see a TAB to our left
   )
          # We are the beginning of a digit

  \d+:    # followed by one or more digits, ending in a colon
  \s+     # then atleast one whitespace
  \Q$c\E  # followed by the item to be colored. \Q...\E quote the $c should
          # it comprise any characters which mean something to the regex.

 (?= \t | $ ) # we see a TAB or the end of line to our right
/x;