Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have some columnized data which I would like to format a little further. It looks like this:

$ getStats | grep ESTABLISHED | column -t
all  tcp  117.54.56.131:80       <-  10.42.100.211:63752   ESTABLISHED:ESTABLISHED
all  tcp  10.42.120.201:63752    ->  219.224.67.112:31180  ->                       137.51.59.141:80       ESTABLISHED:ESTABLISHED
all  tcp  77.221.237.24:443      <-  10.42.100.117:59999   ESTABLISHED:ESTABLISHED

Essentially before I call column -t I want to move the TCP state, 'ESTABLISHED:ESTABLISHED' further on the first and third line so that it is correctly placed with the second line. How would I go about this? Using awk and printf seem a bit messy here. Essentially I want to align all columns and right-align just the last column.

share|improve this question
    
Assuming that getStats is some script based on PHP's getStats it would be more logical to format the printing of its data (the associative array) appropriately in the first place. Or at least make getStats output with unique delimiters. –  Anthon Jun 12 '13 at 5:32
    
That was my first idea Anthon, but seeing as I've had this problem crop up in multiple shell scripts and GNU utilities, I'm just using this as an example for the larger problem. And no, this isn't related to a PHP script. –  atx Jun 12 '13 at 6:42

3 Answers 3

up vote 2 down vote accepted
+50

The real problem here is that the number of columns is not equal. You have some lines with six columns and some with eight.

So what you need to do is to add an empty x-th and y-th field where they are missing (x and y might be 5 and 6, or maybe 3 and 4).

You could do it like this:

F="\\(\\S\\S*\\)\\s*\\s"
# This is 0160, a nonbreaking space
G=" "

| sed -e "s/^$F$F$F$F$F$F*$/\\1 \\2 \\3 \\4 \\5 $G $G \\6/g" \
| column -t

sed identifies those lines with only six fields, and adds two extra fields where appropriate. With the above, I get

all  tcp  117.54.56.131:80     <-  10.42.100.211:63752                         ESTABLISHED:ESTABLISHED
all  tcp  10.42.120.201:63752  ->  219.224.67.112:31180  ->  137.51.59.141:80  ESTABLISHED:ESTABLISHED
all  tcp  77.221.237.24:443    <-  10.42.100.117:59999                         ESTABLISHED:ESTABLISHED
share|improve this answer
    
Thanks, I feel this is the most sane answer, although still not entirely happy about it. I imagine there has to be a way to shift only the last column. –  atx Jun 23 '13 at 4:19

EDIT 1:

One way could be something like:

getStats | grep ESTABLISHED | column -t | sed \
-e 's/\(<-\|->\)[ ]\+/\1 /g' \
-e 's/[ ]\+\([^ ]\+$\)/\t\1/' | column -t -s "   "
                                               ^--- TAB
all  tcp  117.54.56.131:80     <- 10.42.100.211:63752                        ESTABLISHED:ESTABLISHED
all  tcp  10.42.120.201:63752  -> 219.224.67.112:31180  -> 137.51.59.141:80  ESTABLISHED:ESTABLISHED
all  tcp  77.221.237.24:443    <- 10.42.100.117:59999                        ESTABLISHED:ESTABLISHED

First using column -t then removing all consecutive spaces after <- and -> with one space, then separating last column with tab and doing a new column -t -s '<TAB>'

If on command line: -s "Ctrl+VTAB" (aka. tab) as separator for column. Optionally use tr to replace tabs for spaces first.

Setting sed to one op and skipping grep and some other modifications:

getStats | column -t | \
sed '/ESTABLISHED/!d;s/\(<-\|->\) */\1  /g;s/ *\([^ ]*\)$/\t \1/' | \
column -t -s "    "
                ^--- TAB

EDIT 2:

Even though you find awk and printf messy I give this as an option. With this script you can instead say:

getStats | scrip_name ESTABLISHED

One advantage is flexibility to customize etc.

Any way which one look at it one have either to parse the data twice or save meta about data and print at end.

In short what it does is:

  • Record maximum width of each column.
  • Record maximum number of columns.
  • Save each field to an array line-wise.
  • At end print each field but last using max width for that column.
  • Fill up to maximum columns - 1 with spaces.
  • Print last field.

(The split between awk -v pat="$1" ' and rest of code is only due to custom highlight HTML-comment on this page)

#!/bin/bash

# Argument 1 is what to match against.
awk -v pat="$1" '
# Iff match pat.
$0 ~ pat {
    # Highest number of columns.
    if (NF > cols)
        cols = NF
    # Increment number of lines.
    ++nl
    # Number of fileds on this line.
    lines[nl] = NF

    for (i = 1; i <= NF; ++i) {
        # IFF not last field and 
        # width of field is > current width of column, store it in wc_a.
        if (i < NF && (wc = length($i)) > wc_a[i])
            wc_a[i] = wc
        # Save columns in array lines[LINE COLUMN]=FIELD_DATA.
        lines[nl,i] = $i
    }
}

END {
    # Loop lines.
    for (i = 1; i <= nl; ++i) {
        # Print all but last.
        for (j = 1; j < lines[i]; ++j)
            printf("%-*s ", wc_a[j], lines[i,j])
        # Print "missing" columns.
        for (; j < cols; ++j)
            printf("%-*s ", wc_a[j], "")
        # Print last column field.
        printf("%s\n", lines[i,lines[i]])
    }
}
' "$2"
# $2 is either file or empty: expect pipe.

OLD:

Deleted and found here.

share|improve this answer
    
Your first example works pretty well, but doesn't cut it: the columnizing still needs to apply to the other fields, especially the IP addresses. –  atx Jun 12 '13 at 6:41
    
@atx: Yes. I misinterpreted "Essentially I want to right align just the last column." –  Sukminder Jun 12 '13 at 6:57
    
Ah, my mistake. I've edited the post. –  atx Jun 12 '13 at 6:59
    
Though if is is huge data I'd use some other language. A quick test on a 74M file: grep piped to C: 2 seconds, awk: 27 seconds, sed + column: 70 seconds. –  Sukminder Jun 16 '13 at 12:01

Here's a little Perl script to accomplish what you want:

$ getStats | grep ESTABLISHED | \
perl -ne '
chomp @a;
@a = split(" ",$_);
map { print "$_," } @a[0..4];
if ($a[5] !~ m/>/) {
  map { print " , ,$_," } @a[5..$#a];
  print "\n";
} else {
  map { print "$_," } @a[5..$#a];
  print "\n";
}
' | column -t -s ','

Results in this output:

all  tcp  117.54.56.131:80     <-  10.42.100.211:63752                         ESTABLISHED:ESTABLISHED
all  tcp  10.42.120.201:63752  ->  219.224.67.112:31180  ->  137.51.59.141:80  ESTABLISHED:ESTABLISHED
all  tcp  77.221.237.24:443    <-  10.42.100.117:59999                         ESTABLISHED:ESTABLISHED

I took a slightly different approach with your column -t and modified my Perl output so that it introduced a comma "," between each field. So the output before the `column command looks like this:

all,tcp,117.54.56.131:80,<-,10.42.100.211:63752, , ,ESTABLISHED:ESTABLISHED,
all,tcp,10.42.120.201:63752,->,219.224.67.112:31180,->,137.51.59.141:80,ESTABLISHED:ESTABLISHED,
all,tcp,77.221.237.24:443,<-,10.42.100.117:59999, , ,ESTABLISHED:ESTABLISHED,

The column -t -s ',' then coaxes column to split on the separator characters instead, which I find easier to deal with then just plain whitespace.

The introduction of the commas per each line feels a little hacky to me but it does the job, this could likely be streamlined further, but it's a working solution.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.