get only the visible output of a terminal-interpreted string

Question

I am developping a script that will allow me to find easily which columns are "only spaces", by creating a mask showing any column with anything other than a space.

To do this: I print each line "on top of each others" + modify spaces to "right-arrow". (an additional step would also be to ignore spaces seen after a "column farther than the beginning of the last title", but it is out of topic here)

I have trouble with the last step: how to get the resulting string (the output my terminal displays correctly), without every \r, \n, and <Esc>[C (= right arrow) that were used to create/interpret it.

Exemple:

$ PS2=""
$ Esc=$( printf '\033' )
$ Right=$( printf "${Esc}[C" )
$ ps | head -n 2
  PID TTY          TIME CMD
12415 pts/1160 00:00:00 bash
$ ps | head -n 2 | LC_ALL=C tr '!-~' '*'
  *** ***          **** ***
***** ******** ******** ****
$ ps | head -n 2 | LC_ALL=C tr '!-~' '*' | sed -e "s/ /${Right}/g" | while read line; do
    printf "%s\r" "$line"
  done ; printf '\n'
***** ******** ******** ****
  ## the line above is what I am looking for: 
  ## a "mask" of each column that at one point had a non-space character displayed in it
$ ( ps | head -n 2 | tr '!-~' '*' | sed -e "s/ /${Right}/g" | while read line; do
      printf "%s\r" "$line"
    done ; printf '\n'
  ) | cat -ve
^[[C^[[C***^[[C***^[[C^[[C^[[C^[[C^[[C^[[C^[[C^[[C^[[C^[[C****^[[C***^M*****^[[C********^[[C********^[[C****^M$
  ## but of course, the terminal sees (and outputs) this instead, which contains all the "contructing characters"

How could I retrieve in a variable the string "as shown by my terminal", ie:

***** ******** ******** ****

(I tried to output that whole thing into a "while read -e finalstring; do print '[%s]' "$finalstring" ; done , but that finalstring still contains the whole "pre-interpretation" string, not the one "post-terminal interpretation")

A shorted way to ask the same question:

# if:
$ printf "aaaa\rbb\rc\n"
# displays:
cbaa
$ var=$( printf "aaaa\rbb\rc\n" ) # $var will be: "aaaa\rbb\rc\n"
# how can I instead have the resulting "displayed string": "cbaa" in $var ?

Answer 1

It looks like you are just making an extra problem by adding terminal escapes to data that has none to begin with. If all you want is to visualize the blank columns from ps, you can use awk (which it seems you know) to simply accummulate an array indexed by column number, which is all blanks and gets a "*" where necessary. Eg:

awk '
{ len = length($0)
  if(len>max)max = len
  for(i=1;i<=len;i++) if(substr($0,i,1)!=" ") col[i] = "*"
}
END{ for(i=1;i<=max;i++) if(col[i])str = str "*"; else str = str " ";
     print str
}'

Answer 2

A simple ansi parser I used many years ago is the one provided with the Python module pexpect. (Sadly, this is now deprecated in favour of the pyte module, with which I am not familiar.)

This old module still seems to be provided as part of the package. You send it a stream of data and it writes the characters in the appropriate place in a 40x80 array, taking into account the ansi escape sequences.

The following example hooks into the scroll-up action and keeps a one-line array result of 80 characters with a "*" placed wherever a non-space character was scrolled up out of the screen. The str() call on the screen is changed to scroll-up the whole screen to flush it, and then just return this array. Just pipe your data into this program. This is just a start, as you might also want to supress updating the array when spaces overwrite non-space characters, and so on.

#!/usr/bin/python3
# https://unix.stackexchange.com/q/711733/119298
# extend ANSI by noting which cols get non-whitespace
import sys, re
from pexpect.ANSI import ANSI

class KeepScroll(ANSI):
    def __init__(self, r=24,c=80):
        ANSI.__init__(self,r,c)
        ANSI.Log = None
        self.result = [" "] * self.cols

    def scroll_up(self):
        if self.scroll_row_start==1:
            for i, ch in enumerate(self.w[0]):
                if ch!=" ": self.result[i] = "*"
        ANSI.scroll_up(self)

    # scroll up rest of screen
    def __str__(self):
        for i in range(self.rows): self.scroll_up()
        return "".join(self.result)

def parsevt100(data):
    scr = KeepScroll()
    scr.write(data)
    return str(scr)

if __name__ == "__main__":
    print(parsevt100(sys.stdin.read()))