3

Processing a text line-by-line removing leading spaces on each line is easy:

$ LC_ALL=C git ls-files | sed -nE 's:^.*(\.[^./]+)$:\1:p' \
    | sort | uniq -c | sort -snr > lines # create example "lines" file
$ cat lines # "lines" example file
     30 .md
      8 .png
      4 .yml
      1 .css
      1 .gitignore
      1 .ico
      1 .sh
      1 .txt
$ sed -Ee 's/^ +//' lines # removing leading spaces (U+0020)
30 .md
8 .png
4 .yml
1 .css
1 .gitignore
1 .ico
1 .sh
1 .txt

However if only the first line should set the number of spaces to remove of all subsequent lines, how to achieve this? The output would look like:

30 .md
 8 .png
 4 .yml
 1 .css
 1 .gitignore
 1 .ico
 1 .sh
 1 .txt

What I'm trying to achieve is to pipe it to column(1) and make the output more dense but keeping the horizontal spacing across all lines. Simulation:

$ column -x lines | expand -t 8
     30 .md                   8 .png                  4 .yml
      1 .css                  1 .gitignore            1 .ico
      1 .sh                   1 .txt

Right now w/o trimming on the left a lot of space is in use as uniq(1) with the -c option adds them as it does right-justify the numbers (at position 8).

As long as I assume the maximum count is fixed, e.g. at maximum two digits long, I could hard-code it:

sed -Ee 's/^ {5}//' lines | column -x | expand -t 8
30 .md           8 .png          4 .yml          1 .css          1 .gitignore
 1 .ico          1 .sh           1 .txt
1
6

Gnu sed: store the leading whitespace in the hold and then strip away this much amount of leading whitespace from every line. Assuming the lines are sorted as shown.

sed -Ee '
  1{h;s/\S.*//;x;}
  G;s/^(\s*)(.*)\n\1$/\2/
' file

awk '
NR==1 {
  l0=length()
  $1=$1
  re = "^\\s{" l0-length() "}"
}
sub(re, "")+1
' file

perl -lpe '
  $x //= do{/^\s*/g;+pos;};
  $_ = substr($_,$x);
' file
2

I should have been looking into awk(1) earlier. At least I could write a small program that stores the number of leading spaces of the first line and formats every line:

$ awk '
    NR==1 && match($0, /^ */) {p=RLENGTH+1};
    {print(substr($0,p))}
' lines | column -x | expand -t 8
30 .md           8 .png          4 .yml          1 .css          1 .gitignore
 1 .ico          1 .sh           1 .txt
2

Why limit to the first line? As long as you don't need to process gigabytes of data where storing it all in memory would be an issue, you can just save the longest first field, and then use that to format the rest:

$ cat lines 
      4 .yml
      1 .sh
      1 .ico
      1 .gitignore
      1 .css
     30 .md
      1 .txt
      8 .png

And:

$ awk -v l=0 '{ 
                if(length($1)>l){
                    l=length($1)
                } 
                a[$2]=$1
              }
              END{
                for(line in a){
                    printf "%"l"s %s\n",a[line],line
                }
             }' lines 
 8 .png
 1 .ico
 1 .txt
 1 .css
 1 .sh
30 .md
 1 .gitignore
 4 .yml

So:

$ awk -v l=0 '{ if(length($1)>l){l=length($1)} a[$2]=$1}END{for(line in a){printf "%"l"s %s\n",a[line],line}}' lines | 
    column -x | expand -t 8
 8 .png          1 .ico          1 .txt          1 .css          1 .sh
30 .md           1 .gitignore    4 .yml

If you don't need this, and only care about the first line, you could simplify to:

$ perl -pe '/^(\s+)/; $l//=$1; s/^$l//' lines | column -x | expand -t 8
30 .md           4 .yml          1 .sh           1 .ico          1 .gitignore
 1 .css          1 .txt          8 .png
4
  • the first line is just the one with the highest number, so this was not necessary but your answer gives a good example if that is not the case. Very nice! (okay maybe I should have looked into perl first, too).
    – hakre
    Feb 9 at 11:23
  • 1
    @hakre yes, since you're sorting it will be the highest number. I just figured it might be useful to generalize it. The perl will be the simplest for your case.
    – terdon
    Feb 9 at 11:26
  • @terdon adding a 300 .foo to the list I get this misaligned. Feb 9 at 14:02
  • @schrodigerscatcuriosity I suspect you didn't add that as the first line (the perl solution only works if the longest is first). You also need to make sure you add it with the right original alignment in the file. I can reproduce your output either if I dont' add the 300 first or if I don't align it correctly in the input file.
    – terdon
    Feb 9 at 15:12
2

I hope I understand and I'm on the right path:

$ sed 's/^[[:blank:]]//' file | cut -d' ' -f3- | column -x | expand -t 8 | cut -d' ' -f3-
30 .md           8 .png          4 .yml          1 .css          1 .gitignore
 1 .ico          1 .sh           1 .txt

For a scenario like this, for example:

     30 .md
      8 .png
      4 .yml
      1 .css
      1 .gitignore
   4000 .ico
      1 .sh
      1 .txt
    300 .foo
$ sed 's/^[[:blank:]]//' file | cut -d' ' -f3- | column -x | expand -t 8 | cut -d' ' -f3-
30 .md           8 .png          4 .yml          1 .css       4000 .ico      
 1 .gitignore    1 .sh           1 .txt        300 .foo

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.