0

I have folder (data) contain large number of text files. Each text file contain one column of numbers, the column contain 30 numbers. I want to create table that combine the columns in these text files. I used the following command:

  cd data
  paste *.txt > all_data.txt

The output is as follows:

0.834679    0.612341    0.510057    0.580128  .....
1.837894    1.061547    0.961449    1.343096  .....
1.638215    1.024628    1.113596    1.80506  .....
1.596119    0.971842    1.14204     1.73881  .....
1.568142    1.025716    1.217219    1.769668  .....
1.554016    0.977165    1.226769    1.786335  .....
1.543137    1.001812    1.247598    1.840443  .....
1.491823    0.99437     1.28337     1.784271  .....
1.371208    0.931998    1.259749    1.717408  .....
1.390867    0.905996    1.300722    1.739213  .....
1.35813     0.283377    1.307122    1.733058  .....
......

This command combined all the columns in one big spreadsheet. My question: is there any way to paste each column in the text files and make the text file name header to that column. As follows:

text_file1  text_file2  text_file3  text_file4 ....  
0.834679    0.612341    0.510057    0.580128  .....
1.837894    1.061547    0.961449    1.343096  .....
1.638215    1.024628    1.113596    1.80506  .....
1.596119    0.971842    1.14204     1.73881  .....
1.568142    1.025716    1.217219    1.769668  .....
1.554016    0.977165    1.226769    1.786335  .....
1.543137    1.001812    1.247598    1.840443  .....
1.491823    0.99437     1.28337     1.784271  .....
1.371208    0.931998    1.259749    1.717408  .....
1.390867    0.905996    1.300722    1.739213  .....
1.35813     0.283377    1.307122    1.733058  .....
......
3

You can do:

paste_with_header() (
  IFS=$(printf '\t') # or IFS=$'\t' with some shells
  printf '%s\n' "$*"
  paste -- "$@"
)

paste_with_header text_file* > all_data.txt

"$*" is expanded to the list of positional parameters (the arguments to the function) joined with the first character of $IFS, which we set to TAB, the character that paste also uses to join lines of the file.

To remove the file extensions in the header, with zsh:

paste_with_header() (
  printf '%s\n' ${(pj:\t:)@:r}
  paste -- "$@"
)
  • ${@:r} expands to the root name (extension removed) of each positional parameter.
  • j:\t: parameter expansion flag to join on \t. With p that \t is interpreted as a TAB

Or with ksh93, zsh or bash assuming files do have an extension or that no directory in their path components contain dots:

paste_with_header() (
  IFS=$'\t'
  printf '%s\n' "${@%.*}"
  paste -- "$@"
)

${@%.*} removes the shortest trailing part that matches .* off the end of each of the positional parameters (foo.txt becomes foo, but beware that ./foo becomes the empty string).

(those obviously assume the file names don't contain tabulations or newline characters).

  • Thank you so much for the great input. May I kindly ask if you can provide explanation for learning purposes. – user88036 Mar 27 '18 at 16:00
  • I tried the last function in bash. There is no output data? I am sure that I am missing something – user88036 Mar 27 '18 at 16:01
  • 1
    @goro, you still need to invoke that function (like with paste_with_header text_file*.txt > all_data.txt) – Stéphane Chazelas Mar 27 '18 at 16:03
  • Thank you so much for the help and for the explanations! I hope I can give you more than one thumb up ;-) – user88036 Mar 27 '18 at 16:12

protected by Community Mar 28 '18 at 7:01

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?