4

I have some columnized data which I would like to format a little further. It looks like this:

$ getStats | grep ESTABLISHED | column -t
all  tcp  117.54.56.131:80       <-  10.42.100.211:63752   ESTABLISHED:ESTABLISHED
all  tcp  10.42.120.201:63752    ->  219.224.67.112:31180  ->                       137.51.59.141:80       ESTABLISHED:ESTABLISHED
all  tcp  77.221.237.24:443      <-  10.42.100.117:59999   ESTABLISHED:ESTABLISHED

Essentially before I call column -t I want to move the TCP state, 'ESTABLISHED:ESTABLISHED' further on the first and third line so that it is correctly placed with the second line. How would I go about this? Using awk and printf seem a bit messy here. Essentially I want to align all columns and right-align just the last column.

2
  • Assuming that getStats is some script based on PHP's getStats it would be more logical to format the printing of its data (the associative array) appropriately in the first place. Or at least make getStats output with unique delimiters.
    – Anthon
    Jun 12, 2013 at 5:32
  • That was my first idea Anthon, but seeing as I've had this problem crop up in multiple shell scripts and GNU utilities, I'm just using this as an example for the larger problem. And no, this isn't related to a PHP script.
    – atx
    Jun 12, 2013 at 6:42

3 Answers 3

3

EDIT 1:

One way could be something like:

getStats | grep ESTABLISHED | column -t | sed \
-e 's/\(<-\|->\)[ ]\+/\1 /g' \
-e 's/[ ]\+\([^ ]\+$\)/\t\1/' | column -t -s "   "
                                               ^--- TAB
all  tcp  117.54.56.131:80     <- 10.42.100.211:63752                        ESTABLISHED:ESTABLISHED
all  tcp  10.42.120.201:63752  -> 219.224.67.112:31180  -> 137.51.59.141:80  ESTABLISHED:ESTABLISHED
all  tcp  77.221.237.24:443    <- 10.42.100.117:59999                        ESTABLISHED:ESTABLISHED

First using column -t then removing all consecutive spaces after <- and -> with one space, then separating last column with tab and doing a new column -t -s '<TAB>'

If on command line: -s "Ctrl+VTAB" (aka. tab) as separator for column. Optionally use tr to replace tabs for spaces first.

Setting sed to one op and skipping grep and some other modifications:

getStats | column -t | \
sed '/ESTABLISHED/!d;s/\(<-\|->\) */\1  /g;s/ *\([^ ]*\)$/\t \1/' | \
column -t -s "    "
                ^--- TAB

EDIT 2:

Even though you find awk and printf messy I give this as an option. With this script you can instead say:

getStats | scrip_name ESTABLISHED

One advantage is flexibility to customize etc.

Any way which one look at it one have either to parse the data twice or save meta about data and print at end.

In short what it does is:

  • Record maximum width of each column.
  • Record maximum number of columns.
  • Save each field to an array line-wise.
  • At end print each field but last using max width for that column.
  • Fill up to maximum columns - 1 with spaces.
  • Print last field.

(The split between awk -v pat="$1" ' and rest of code is only due to custom highlight HTML-comment on this page)

#!/bin/bash

# Argument 1 is what to match against.
awk -v pat="$1" '
# Iff match pat.
$0 ~ pat {
    # Highest number of columns.
    if (NF > cols)
        cols = NF
    # Increment number of lines.
    ++nl
    # Number of fileds on this line.
    lines[nl] = NF

    for (i = 1; i <= NF; ++i) {
        # IFF not last field and 
        # width of field is > current width of column, store it in wc_a.
        if (i < NF && (wc = length($i)) > wc_a[i])
            wc_a[i] = wc
        # Save columns in array lines[LINE COLUMN]=FIELD_DATA.
        lines[nl,i] = $i
    }
}

END {
    # Loop lines.
    for (i = 1; i <= nl; ++i) {
        # Print all but last.
        for (j = 1; j < lines[i]; ++j)
            printf("%-*s ", wc_a[j], lines[i,j])
        # Print "missing" columns.
        for (; j < cols; ++j)
            printf("%-*s ", wc_a[j], "")
        # Print last column field.
        printf("%s\n", lines[i,lines[i]])
    }
}
' "$2"
# $2 is either file or empty: expect pipe.

OLD:

Deleted and found here.

4
  • Your first example works pretty well, but doesn't cut it: the columnizing still needs to apply to the other fields, especially the IP addresses.
    – atx
    Jun 12, 2013 at 6:41
  • @atx: Yes. I misinterpreted "Essentially I want to right align just the last column."
    – Runium
    Jun 12, 2013 at 6:57
  • Ah, my mistake. I've edited the post.
    – atx
    Jun 12, 2013 at 6:59
  • Though if is is huge data I'd use some other language. A quick test on a 74M file: grep piped to C: 2 seconds, awk: 27 seconds, sed + column: 70 seconds.
    – Runium
    Jun 16, 2013 at 12:01
2
+50

The real problem here is that the number of columns is not equal. You have some lines with six columns and some with eight.

So what you need to do is to add an empty x-th and y-th field where they are missing (x and y might be 5 and 6, or maybe 3 and 4).

You could do it like this:

F="\\(\\S\\S*\\)\\s*\\s"
# This is 0160, a nonbreaking space
G=" "

| sed -e "s/^$F$F$F$F$F$F*$/\\1 \\2 \\3 \\4 \\5 $G $G \\6/g" \
| column -t

sed identifies those lines with only six fields, and adds two extra fields where appropriate. With the above, I get

all  tcp  117.54.56.131:80     <-  10.42.100.211:63752                         ESTABLISHED:ESTABLISHED
all  tcp  10.42.120.201:63752  ->  219.224.67.112:31180  ->  137.51.59.141:80  ESTABLISHED:ESTABLISHED
all  tcp  77.221.237.24:443    <-  10.42.100.117:59999                         ESTABLISHED:ESTABLISHED
1
  • Thanks, I feel this is the most sane answer, although still not entirely happy about it. I imagine there has to be a way to shift only the last column.
    – atx
    Jun 23, 2013 at 4:19
1

Here's a little Perl script to accomplish what you want:

$ getStats | grep ESTABLISHED | \
perl -ne '
chomp @a;
@a = split(" ",$_);
map { print "$_," } @a[0..4];
if ($a[5] !~ m/>/) {
  map { print " , ,$_," } @a[5..$#a];
  print "\n";
} else {
  map { print "$_," } @a[5..$#a];
  print "\n";
}
' | column -t -s ','

Results in this output:

all  tcp  117.54.56.131:80     <-  10.42.100.211:63752                         ESTABLISHED:ESTABLISHED
all  tcp  10.42.120.201:63752  ->  219.224.67.112:31180  ->  137.51.59.141:80  ESTABLISHED:ESTABLISHED
all  tcp  77.221.237.24:443    <-  10.42.100.117:59999                         ESTABLISHED:ESTABLISHED

I took a slightly different approach with your column -t and modified my Perl output so that it introduced a comma "," between each field. So the output before the `column command looks like this:

all,tcp,117.54.56.131:80,<-,10.42.100.211:63752, , ,ESTABLISHED:ESTABLISHED,
all,tcp,10.42.120.201:63752,->,219.224.67.112:31180,->,137.51.59.141:80,ESTABLISHED:ESTABLISHED,
all,tcp,77.221.237.24:443,<-,10.42.100.117:59999, , ,ESTABLISHED:ESTABLISHED,

The column -t -s ',' then coaxes column to split on the separator characters instead, which I find easier to deal with then just plain whitespace.

The introduction of the commas per each line feels a little hacky to me but it does the job, this could likely be streamlined further, but it's a working solution.

This site is temporarily in read-only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .