2

I have a text file on a Linux machine with two columns:

  • Column 1 = id_no (most are 5, with some 6 digits long);
  • Column 2 = genetic_markers (all are 50674 digits long);
12345 0102010205
54322 2221110051
123456 1122011510

I want to change the file to look as follows:

 12345 0 1 0 2 0 1 0 2 0 5
 54322 2 2 2 1 1 1 0 0 5 1
123456 1 1 2 2 0 1 1 5 1 0
  1. How do I change the first column so that it aligns to the right (as numbers do)?

  2. Can someone please help me with the most reliable way to change the second column with spaces between the digits? Please give an explanation of the elements of the code and what it does.

Thanks

3
  • Do you know the length of the longest number of the first field? Or do we need to detect it? Are you really, really, 100% sure you want the numbers to be right aligned? Yes, it might make it slightly easier for a human to read, but it also makes it (slightly) harder for a computer to read since some tools might get confused by the leading spaces.
    – terdon
    Jan 26 at 19:20
  • @terdon I hear what you are saying and I agree. I am following a manual with examples of a program, and because I am struggling I want to follow it exactly. But it might not make a difference. Jan 26 at 20:09
  • please update the question with your coding attempts
    – markp-fuso
    Jan 26 at 20:37

5 Answers 5

7

With perl:

$ perl -lane 'printf "%6s %s\n", $F[0], join " ", split "", $F[1]' <your-file
 12345 0 1 0 2 0 1 0 2 0 5
 54322 2 2 2 1 1 1 0 0 5 1
123456 1 1 2 2 0 1 1 5 1 0

With -lane (-n reads input one record at a time and runs the -expression with the record in $_, -a for awk splits the line into @Fields, -l removes the line delimiter from the record) , perl behaves like awk.

Here, we use printf to left-pad the first field to a length of 6 with spaces, followed by the joining with spaces of the second field split into its character constituents.

3
  • Based on the what @terdon said, what would the version be where we leave field 1 values left-aligned. Then I can try both methods. Jan 26 at 20:24
  • PS. How do you it so that the results is a new file: myfile > newfile ? Jan 26 at 20:32
  • 1
    For left-aligned, use %-6d instead of %6d. To redirect the output to newfile, yes, > newfile. Jan 26 at 20:46
5

In order to right-align, you need to either find the length of the longest number in the file, or just pick a large number and use that. For example, you can pad with 10 spaces:

$ printf '%d\n' 123
123
$ printf '%10d\n' 123
       123

If that approach is good enough, you could do something like:

$ awk '{ gsub(/./," &",$2); printf "%10d%s\n",$1,$2}' file 
     12345 0 1 0 2 0 1 0 2 0 5
     54322 2 2 2 1 1 1 0 0 5 1
    123456 1 1 2 2 0 1 1 5 1 0

There are only two things happening here:

  • gsub(/./," &",$2);: the gsub (global substitution) function will replace all matches of the regular expression you give it (here, we give it just a . meaning "any character") with whatever you give as the replacement. The & has a special meaning and means "whatever was matched by the regular expression", so giving & as the replacement has the effect of inserting one space before every character. The final argument is the input, and here we are giving it the second field, $2.
  • printf "%10d %s\n",$1,$2: we are using printf to print a formatted string. %10d means "print the number I give you and pad it with 10 spaces", and %s means "print this string". So, we tell it to print the first field padded by 10 spaces, and then the second field which has been modified by gsub.

If you need to pad only the minimum, you need to read the file twice. First to get the length of the longest first field:

$ awk -v max=0 '{ if(length($1) > max){ max=length($1) }} END{print max}' file 
6

Armed with this, you can be more specific:

$ awk '{ k=gsub(/./," &",$2); printf "%6d%s\n",$1,$2}' file 
 12345 0 1 0 2 0 1 0 2 0 5
 54322 2 2 2 1 1 1 0 0 5 1
123456 1 1 2 2 0 1 1 5 1 0
6
  • +1 for the super-didactic way of presenting this answer. Nevertheless I would include half a sentence explaining how -v max=0makes explicit the fact that there is a (external) variable, "max", initialized to zero, but you really don't need it to be external + "max" is implicitly set to zero every time Awk starts executing.
    – Cbhihe
    Jan 27 at 9:35
  • @Cbhihe I think that is true of gawk but not all awks. If I remember correctly, some will complain and fail if you try to use an unset variable.
    – terdon
    Jan 27 at 12:02
  • 1
    All awks will initialize an unset variable to the value zero-or-null, none will complain about using an unset variable. That's how you can tell if a variable is set or not - if ( (foo == 0) && (foo == "") ) print "foo is not set"; else print "foo is set". Only a variable that hasn't been set will be equal to both zero and null. It has that value because awk decides if a scalar variable is a string or a number based on the first time that variable is set so if it hasn't been set yet then it's got attributes of both a string (null) and a number (0).
    – Ed Morton
    Jan 27 at 12:18
  • 1
    @Cbhihe max isn't an external variable, it's internal to awk, just like it it was set in awk '{BEGIN{max=0} ...'. If we had max=0; awk -v max="$max" '...' or max=0 awk '{ if(length($1) > ENVIRON["max"]) } ...' THEN there'd be an external (shell) variable named max and, in the first case but not the second, also an internal awk variable of the same name. See how-do-i-use-shell-variables-in-an-awk-script for more on shell vs awk variables.
    – Ed Morton
    Jan 27 at 12:25
  • @EdMorton: yes, I was sloppy ... although I tried not to be writing "external" between parentheses. It's not external but using the Awk option -v max=0 is a mechanism usually used to pass shell (i.e. "external") variable into Awk space.
    – Cbhihe
    Jan 27 at 13:37
4

Using any awk and GNU column (for -R):

$ awk '{gsub(/./," &",$2)} 1' file | column -tR1
 12345  0  1  0  2  0  1  0  2  0  5
 54322  2  2  2  1  1  1  0  0  5  1
123456  1  1  2  2  0  1  1  5  1  0

add -o' ' if you really care about the spaces between fields:

$ awk '{gsub(/./," &",$2)} 1' file | column -o' ' -t -R1
 12345 0 1 0 2 0 1 0 2 0 5
 54322 2 2 2 1 1 1 0 0 5 1
123456 1 1 2 2 0 1 1 5 1 0
1
  • @Cbhihe thanks! I suspect it's still parsing the input twice, I just moved one of those iterations from awk to column which hopefully does that more efficiently (certainly more concisely).
    – Ed Morton
    Jan 27 at 13:52
4

Previous answers by @terdon and @StéphaneChazelas are fine, but bunching together the two stream parses (first to obtain the max string length of column 1 values, second to use that maximum as formatting parameter) might be of interest. Doing so based on @terdon's answer and notation gives:

awk 'NR==FNR { if( length($1)>max ) { max=length($1) }; next }
             { gsub(/./," &",$2); printf "%*d%s\n",max,$1,$2 }' myfile myfile

Note how:

  • the previously variable "max" does not need initialization to 0,
  • "k" in k=gsub(...) is not needed,
  • the maximum string length, max, is used to substitute the * in the printf "%*d%s\n"... format
  • two parses of the same file "myfile" reamin, whereby (NR==FNR{...; next}) relates to the first parse only, while the second {...} command block relates to the second one only.
0
4

Using Raku (formerly known as Perl_6)

~$ raku -ne '.split(" ") andthen put sprintf("%6d", .[0]), .[1].comb;'  file 

#OR

~$ raku -ne '.words andthen put sprintf("%6d", .[0]), .[1].comb;'  file

Raku is a programming language in the Perl-family. Above uses Raku's -ne non-autoprinting linewise flags. The -n flag removes the delimiter from the line terminus by default. You can then print omitting a trailing newline (by default), or put, which adds a trailing newline (think of put standing for print-using-terminator).

In the first answer the line is explicitly .split on single-whitespace (short for $_.split). In the second answer Raku's .words routine is used to split on whitespace. After this the conjunctive andthen reloads $_ so that each column can be formatted for output. The first column (i.e. .[0]) is formatted using sprintf while the second column (i.e. .[1]) is combed into single characters and returned.


Note: If indeed "IDs" are (decimal) unsigned integers, then within sprintf you can use u instead of d, as in sprintf("%6u", …).


Sample Input:

12345 0102010205
54322 2221110051
123456 1122011510

Sample Output:

 123450 1 0 2 0 1 0 2 0 5
 543222 2 2 1 1 1 0 0 5 1
1234561 1 2 2 0 1 1 5 1 0

Note: to avoid throwing an error if perchance your file contains blank lines, you can add an if conditional to remove blank lines:

~$ raku -ne 'if .chars { .words andthen put sprintf("%6d", .[0]), .[1].comb};'  file

#OR

~$ raku -ne 'if $_ .= words {put sprintf("%6d", .[0]), .[1].comb};'  file

To retain blank lines, you can use Raku's Ternary operator:

~$ raku -ne '.chars ??  ( .split(" ") andthen put sprintf( "%6d", .[0]), .[1].comb) !! "".put;'  file

#OR

~$ raku -ne '$_ .= split(" ", :skip-empty) ?? (put sprintf( "%6d", .[0]), .[1].comb) !! "".put;'  file 

https://docs.raku.org/routine/sprintf
https://docs.raku.org/routine/%3F%3F%20%21%21
https://raku.org

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .