1

I have n files (call them input1, input2, and so on) with similar data and I wish to make a new file (call it out) that contains the 2nd column of these files. If I use

awk '{print $2}' input{1..n} >> out

then I get a single column with all the entries from the 2nd column of the input files. What can I do to have different columns for different files, as in $1 in out = $2 of input1, $2 in out = $2 of input2, $3 in out = $2 of input3,....., $n in out = $2 of inputn?

  • Are all files of the same length, i.e. do they all always have the same number of rows, and is this number known in advance? – Kusalananda Feb 19 at 14:23
  • @Kusalananda yes they're all of the same length and the #rows and #columns are known. – Hitanshu Sachania Feb 19 at 14:26
1

You could do the whole thing in a BEGIN statement using getline

awk '
  BEGIN {
    while(1) {
      line = sep = ""
      for (i = 1; i < ARGC; i++) {
        if ((getline < ARGV[i]) <= 0) exit
        line = line sep $2
        sep = OFS
      }
      print line
    }
  }' input{1..n} > out
  • Is there any way we can use ORS? I was trying that but it doesn't work with multiple files. – Prvt_Yadv Feb 19 at 14:04
  • @PRY, there was a missing loop. It should be fixed now. – Stéphane Chazelas Feb 19 at 14:04
  • Sorry, I was asking in general manner, not related to your answer. – Prvt_Yadv Feb 19 at 14:05
  • 1
    @HitanshuSachania, getline retrieves one record from the given input file and return 0 upon end-of-file or a negative number upon error in which case we exit (so we stop as soon as we've reached the end of any of the input files so the number of records in the output file is that of the input file with fewest records). – Stéphane Chazelas Feb 19 at 14:51
  • 1
    @HitanshuSachania, not it builds one line of output (in line) at a time, as opposed to storing all the lines of output in an array and printing it at the end. We could skip storing the line in line and print the fields as they come, but that would cause problem for the last record if not all input files have the same number of records. – Stéphane Chazelas Feb 19 at 15:48
1

You could construct a paste command to put all the second columns together:

cmd="paste"
for x in input{1..n}; do
   cmd="$cmd <(awk '{print \$2;}' $x)"
done
echo $cmd
eval $cmd
1

using this post as reference

awk '{a[FNR] = a[FNR]" " $2}END{for(i=1;i<=FNR;i++) print a[i]}' input{1..n}

an array holds each line from different files

FNR number of records read in current input file, set to zero at begining of each file.

END{for(i=1;i<FNR;i++) print a[i]} 

prints the content of array a on END of file

  • What is the purpose of the double quotes before $2 and how does the array a store just the 2nd column? – Hitanshu Sachania Feb 19 at 14:14
  • Note that it stores the whole output in memory before starting to print it. – Stéphane Chazelas Feb 19 at 14:16
  • I changed the lstaro of the loop froom 0 to 1 – Emilio Galarraga Feb 19 at 14:23
  • 1
    the double quotes before $2 defile the delimiter between columns in the output file and the script is passing the value in $2 to a[FNR] element of the array – Emilio Galarraga Feb 19 at 14:27
  • Use {a[FNR]=!a[FNR]?$2:a[FNR]" "$2} to eleminate white space in front of each line... – pLumo Feb 19 at 14:35
1

I would use the pr tool, which is designed to columnize data:

awk '{print $2}' input{1..n} | pr -t --columns=n > out

This assumes each file has the same number of lines.

  • The best answer for this scenario. – Rakesh Sharma Feb 20 at 1:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.