Is there a simple command for outputting tab-delimited columns?

Question

E.g. I have a file (produced with echo -e "var1\tvar2\t\var3\tvar4" > foo) that are output as:

$ cat foo
case    elems   meshing nlsys
uniform 2350    0.076662        2.78
non-conformal   348     0.013332        0.55
scale   318     0.013333        0.44
smarter 504     0.016666        0.64
submodel        360     .009999 0.40
unstruct-quad   640     0.019999        0.80
unstruct-tri    1484    0.01    0.88

I'd prefer the output like this (here I used vim and :set tabstop=14):

case          elems         meshing       nlsys
uniform       2350          0.076662      2.78
non-conformal 348           0.013332      0.55
scale         318           0.013333      0.44
smarter       504           0.016666      0.64
submodel      360           .009999       0.40
unstruct-quad 640           0.019999      0.80
unstruct-tri  1484          0.01          0.88

I can get the same functionality with cat if I use $ tabs=15 in bash (see this question). Is there a program that does this kind of formatting automatically? I don't want to experiment with the tabs value before cating a file.

Answer 1

I usually use the column program for this, it's in a package called bsdmainutils on Debian:

column -t foo

Output:

case           elems  meshing   nlsys
uniform        2350   0.076662  2.78
non-conformal  348    0.013332  0.55
scale          318    0.013333  0.44
smarter        504    0.016666  0.64
submodel       360    .009999   0.40
unstruct-quad  640    0.019999  0.80
unstruct-tri   1484   0.01      0.88

Excerpt from column(1) on my system:

...

-t      Determine the number of columns the input contains and create a
        table.  Columns are delimited with whitespace, by default, or
        with the characters supplied using the -s option.  Useful for
        pretty-printing displays.

...

Answer 2

Several options:

var1=uniform var2=2350 var3=0.076662 var4=2.78

printf '%-15s %-10s %-12s %s\n' \
  case elems messing nlsys \
  "$var1" "$var2" "$var3" "$var4"

printf '%s\t%s\t%s\t%s\n' \
  case elems messing nlsys \
  "$var1" "$var2" "$var3" "$var4" |
  expand -t 15,25,37

printf '%s\t%s\t%s\t%s\n' \
  case elems messing nlsys \
  "$var1" "$var2" "$var3" "$var4" |
  column -t -s $'\t'

column is a non-standard command, some implementations/versions don't support the -s option. It computes the width of the column based on the input, but that means that it can only start displaying once all the input has been fed to it. $'...' is ksh93 syntax also found in zsh and bash.

With zsh:

values=(
  case elems messing nlsys
  "$var1" "$var2" "$var3" "$var4"
)
print -arC4 -- "$values[@]"

Answer 3

Another tool that can do this is tsv-pretty from eBay's TSV Utilities (disclaimer: I'm the author). It takes the additional step of lining up numeric fields on the decimal point. For example:

$ tsv-pretty foo
case           elems   meshing  nlsys
uniform         2350  0.076662   2.78
non-conformal    348  0.013332   0.55
scale            318  0.013333   0.44
smarter          504  0.016666   0.64
submodel         360   .009999   0.40
unstruct-quad    640  0.019999   0.80
unstruct-tri    1484  0.01       0.88

There are several formatting options. For example, -u underlines the header and -f formats the floats in a field similarly for readability:

$ tsv-pretty foo -f -u
case           elems   meshing  nlsys
----           -----   -------  -----
uniform         2350  0.076662   2.78
non-conformal    348  0.013332   0.55
scale            318  0.013333   0.44
smarter          504  0.016666   0.64
submodel         360  0.009999   0.40
unstruct-quad    640  0.019999   0.80
unstruct-tri    1484  0.010000   0.88

More info is available in the tsv-pretty reference.

Answer 4

One problem with column -ts$'\t' is that it removes empty fields, so it effectively removes tabs from the start of lines and replaces two or more consecutive tabs with a single tab:

$ echo $'\tcol1\tcol2\tcol3\nrow1\taa\t1\t100\nrow2\t\t\t200'>test.tsv
$ tr \\t ,<test.tsv
,col1,col2,col3
row1,aa,1,100
row2,,,200
$ column -ts$'\t' test.tsv
col1  col2  col3
row1  aa    1     100
row2  200

Therefore I use this gawk function instead (it relies on arrays of arrays which are not supported by nawk which is /usr/bin/awk on macOS):

$ tab(){ gawk '{if(NF>m)m=NF;for(i=1;i<=NF;i++){a[NR][i]=$i;l=length($i);if(l>b[i])b[i]=l}}END{for(h in a){for(i=1;i<=m;i++)printf(i==m?"%s\n":"%-"(b[i]+n)"s",a[h][i])}}' "${1+FS=$1}" "n=${2-1}";}
$ tab \\t<test.tsv
     col1 col2 col3
row1 aa   1    100
row2           200

A second option is to use rs (which is a BSD utility that also comes with macOS):

$ x=$(cat test.tsv);rs -c -z $(wc -l<<<"$x")<<<"$x"
      col1  col2  col3
row1  aa    1     100
row2              200

In rs, -c changes the input column separator and a lone -c sets the input column separator to a tab. -z sets the width of each column to the width of the longest entry of the column instead of making all columns the same width. If some lines have fewer columns than the first line, add the -n option which adds empty columns to pad out lines which have fewer columns than the first line.

A third option is to use csvtk:

$ csvtk -t pretty test.tsv
       col1   col2   col3
----   ----   ----   ----
row1   aa     1      100
row2                 200
$ csvtk -t pretty -s' ' test.tsv|sed 2d
     col1 col2 col3
row1 aa   1    100
row2           200

The -t option uses tab as a field separator but it doesn't disable CSV-style treatment of double quotes. However -l (--lazy-quote) allows including unescaped double quotes in the input:

$ echo $'1"\t2\n3\t4'|csvtk -t pretty
[ERRO] parse error on line 1, column 2: bare " in non-quoted-field
$ echo $'1"\t2\n3\t4'|csvtk -tl pretty
1"   2
--   -
3    4

Answer 5

Another quick and dirty way around this is to use the tabs command, which is included with ncurses and already installed on a high percentage of *nix systems.

Personally I like this solution because it's super fast and easy to get mostly right.

Based on the example tab delimited file @sebastian provided:

Just run:

tabs 1,20,35,50

Once you're done, probably best to restore tabs to the standard hardware default...

tabs -8

(Yes, I think it's very strange to explicitly set tabs to a -8, but that was my take on the best solution from reading man tabs, and that is the hardware default on every terminal I've seen)

To build a more complete solution, I might would use stty size to check the terminal width, and calculate the width of variable length fields as a % of the terminal width, and truncate long columns in an application sensible way.

Answer 6

The question was about outputting tab delimited columns.

So the correct answer is a small adaptation of the answer of @nisetama. I added the -C$'\t' option which sets output formatting.

x=$(cat foo2); rs -C$'\t' $(wc -l <<<"$x") <<<"$x"

Kudo's to @nisetama though :)

Answer 7

function printTable()
{
    local -r delimiter="${1}"
    local -r data="$(removeEmptyLines "${2}")"

    if [[ "${delimiter}" != '' && "$(isEmptyString "${data}")" = 'false' ]]
    then
        local -r numberOfLines="$(wc -l <<< "${data}")"

        if [[ "${numberOfLines}" -gt '0' ]]
        then
            local table=''
            local i=1

            for ((i = 1; i <= "${numberOfLines}"; i = i + 1))
            do
                local line=''
                line="$(sed "${i}q;d" <<< "${data}")"

                local numberOfColumns='0'
                numberOfColumns="$(awk -F "${delimiter}" '{print NF}' <<< "${line}")"

                # Add Line Delimiter

                if [[ "${i}" -eq '1' ]]
                then
                    table="${table}$(printf '%s#+' "$(repeatString '#+' "${numberOfColumns}")")"
                fi

                # Add Header Or Body

                table="${table}\n"

                local j=1

                for ((j = 1; j <= "${numberOfColumns}"; j = j + 1))
                do
                    table="${table}$(printf '#| %s' "$(cut -d "${delimiter}" -f "${j}" <<< "${line}")")"
                done

                table="${table}#|\n"

                # Add Line Delimiter

                if [[ "${i}" -eq '1' ]] || [[ "${numberOfLines}" -gt '1' && "${i}" -eq "${numberOfLines}" ]]
                then
                    table="${table}$(printf '%s#+' "$(repeatString '#+' "${numberOfColumns}")")"
                fi
            done

            if [[ "$(isEmptyString "${table}")" = 'false' ]]
            then
                echo -e "${table}" | column -s '#' -t | awk '/^\+/{gsub(" ", "-", $0)}1'
            fi
        fi
    fi
}

function removeEmptyLines()
{
    local -r content="${1}"

    echo -e "${content}" | sed '/^\s*$/d'
}

function repeatString()
{
    local -r string="${1}"
    local -r numberToRepeat="${2}"

    if [[ "${string}" != '' && "${numberToRepeat}" =~ ^[1-9][0-9]*$ ]]
    then
        local -r result="$(printf "%${numberToRepeat}s")"
        echo -e "${result// /${string}}"
    fi
}

function isEmptyString()
{
    local -r string="${1}"

    if [[ "$(trimString "${string}")" = '' ]]
    then
        echo 'true' && return 0
    fi

    echo 'false' && return 1
}

function trimString()
{
    local -r string="${1}"

    sed 's,^[[:blank:]]*,,' <<< "${string}" | sed 's,[[:blank:]]*$,,'
}

SAMPLE RUNS

$ cat data-1.txt
HEADER 1,HEADER 2,HEADER 3

$ printTable ',' "$(cat data-1.txt)"
+-----------+-----------+-----------+
| HEADER 1  | HEADER 2  | HEADER 3  |
+-----------+-----------+-----------+

$ cat data-2.txt
HEADER 1,HEADER 2,HEADER 3
data 1,data 2,data 3

$ printTable ',' "$(cat data-2.txt)"
+-----------+-----------+-----------+
| HEADER 1  | HEADER 2  | HEADER 3  |
+-----------+-----------+-----------+
| data 1    | data 2    | data 3    |
+-----------+-----------+-----------+

$ cat data-3.txt
HEADER 1,HEADER 2,HEADER 3
data 1,data 2,data 3
data 4,data 5,data 6

$ printTable ',' "$(cat data-3.txt)"
+-----------+-----------+-----------+
| HEADER 1  | HEADER 2  | HEADER 3  |
+-----------+-----------+-----------+
| data 1    | data 2    | data 3    |
| data 4    | data 5    | data 6    |
+-----------+-----------+-----------+

$ cat data-4.txt
HEADER
data

$ printTable ',' "$(cat data-4.txt)"
+---------+
| HEADER  |
+---------+
| data    |
+---------+

$ cat data-5.txt
HEADER

data 1

data 2

$ printTable ',' "$(cat data-5.txt)"
+---------+
| HEADER  |
+---------+
| data 1  |
| data 2  |
+---------+

REF LIB at: https://github.com/gdbtek/linux-cookbooks/blob/master/libraries/util.bash