Pass colors from ls through pipe to awk 'print' statement

Question

This is a follow-up to my question from yesterday, Show sum of file sizes in directory listing.

Thanks to Zero Piraeus and a point in the right direction by Mauritz Hansen, I now have

function dir() {
        ls -FaGl "${@}" | awk '{print; total += $4}; END {print "\t\ttotal: ",total/1024,"KB"}'
}

in my .bash_profile, and it works great. However, at least on Linux (I haven't had a chance to try it on OSX yet), where I'm sshing in using PuTTY on XP, my dir colors are now gone. Is there some way of passing the color codes through the pipe to the awk print statement?

Update

Thanks to Sukminder's answer, adding --color=always does the trick, as it had been set to auto before. However, I now get a blank space between the end of the directory listing and the total line:

[19:30:58 mattdmo@server ~/webapps/django15 ] $ dir
drwxr-xr-x  7 mattdmo 4096 Mar 24 20:28 ./
drwxr-xr-x 17 root    4096 Mar 18 20:15 ../
drwxr-xr-x  7 mattdmo 4096 Mar 14 14:57 apache2/
drwxr-xr-x  3 mattdmo 4096 Mar 14 14:57 bin/
drwxr-xr-x  2 mattdmo 4096 Mar 24 20:10 lib/
drwxr-xr-x  3 mattdmo 4096 Mar 14 14:57 myproject/
drwxrwxr-x  3 mattdmo 4096 Mar 24 20:28 pigimal/

                total:  28 KB
[19:30:59 mattdmo@server ~/webapps/django15 ] $

Any suggestions on fixing this?

Answer 1

By using ls --color=always the colors should be preserved. I'm unsure if it is gnu specific though. [Edit from OP: yes, it is.]

The case is that ls detect whether output is to tty or not.

If it is not it normally do not print colors and do not translate "unprintable characters" to question marks. This can be added by the -q option.

More on the subject: Number of lines outputted by ls.

---

gnu coreutils source ls; ref. on some of the colors data:

• enum definition.
• link to option word
• option definition

---

Edit:

For gnu coreutils ls: --color=always
For OS X's ls: set environment variable CLICOLOR_FORCE and G means color. The core is conforming to ls IEEE Std 1003.1-2001. Other options are extensions that wary between implementations.

OK. Had a deeper look at this. Though from what I read around people also install gnu coreutils on OS X, I guess you perhaps use the original.

The OS X man pages for ls state:

CLICOLOR_FORCE  
    Color sequences are normally disabled if the output isn't directed to a
    terminal.  This can be overridden by setting this flag.  The TERM 
    variable still needs to reference a color capable terminal however
    otherwise it is not possible to determine which color sequences to use.

From the source:

Mac OS X 10.8.2 Source -> file_cmds-220.7 -> ls/

From that code it is also quickly clear that one can set

CLICOLOR_FORCE

to force color.

As it first parses arguments and setenv color on G:

    case 'G':
        setenv("CLICOLOR", "", 1);
        break;

And after argv parsing:

/* CLICOLOR is set    AND    is a tty            OR */
if (getenv("CLICOLOR") && (isatty(STDOUT_FILENO) || 
etenv("CLICOLOR_FORCE")))
/*  force cli color is set */

By next section if TERM is set, color is used:

if (tgetent(termcapbuf, getenv("TERM")) == 1) {

echo $TERM should yield a color capable terminal. But if you get colors by normal this should be the case.

---

Edit to last update:

Not sure then. It is strange. Not only the extra line but also that you do not have any total ... at the start. You could start out by checking the output of something like this:

function dir() {
/bin/ls -FaGl "${@}" | awk '
    function chr2hex(chr) {
        return sprintf("%02x", index(i2x, chr));
    }
    function str2hex(str) {
        r = ""
        for (i = 1; i <= length(str); ++i)
            r = r chr2hex(substr(str, i, 1));
        return r;
    }
    BEGIN {
        i2x = ""
        for (i = 0; i < 256; ++i)
            i2x = sprintf("%s%c", i2x, i);
        printf("FNR        : %d\n", FNR);
        printf("FIELDWIDTHS: \"%s\"\n", FIELDWIDTHS);
        printf("FS         : \"%s\"\n", str2hex(FS));
        printf("RS         : \"%s\"\n", str2hex(RS));
        printf("OFS        : \"%s\"\n", str2hex(OFS));
        printf("ORS        : \"%s\"\n", str2hex(OFS));
    }
    {
        printf("%2d:%2d:%2d \"%s\"\n", FNR, NR, NF, $0);
        total += $4;
    }
    END {
        printf("%21s: %d %s\n", "total", total / 1024, "KiB");
        printf("FNR        : %d\n", FNR);
        printf("From       : \"%s\"\n", FILENAME);
    }
'
}

Should give you something like – (From could be "-" by e.g. gawk):

$ ./dir testdir
FNR        : 0
FIELDWIDTHS: ""
FS         : "20"
RS         : "0a"
OFS        : "20"
ORS        : "20"
 1: 1: 2 "total 24"
 2: 2: 8 "drwxrwxr-x 6 mattdmo 4096 Apr 17 21:46 ./"
 3: 3: 8 "drwxrwxr-x 8 mattdmo 4096 Apr 18 10:47 ../"
 4: 4: 8 "drwxrwxr-x 2 mattdmo 4096 Apr 17 21:46 ab1/"
 5: 5: 8 "drwxrwxr-x 2 mattdmo 4096 Apr 17 21:46 ab2/"
 6: 6: 8 "drwxrwxr-x 2 mattdmo 4096 Apr 17 21:46 ab3/"
 7: 7: 8 "drwxrwxr-x 2 mattdmo 4096 Apr 17 21:46 ab4/"
                total: 24 KB
FNR        : 7
From       : ""