2

I wanted to organize my dataset by copying the a column from a number of text file and append the file name in the first row of the new files. Here is sample of my data

file1

a   a   b   b
1   2   3   4

file2

c   d   e   f
g   h   i   g

. . .

file3

11  12  23  12
2   4   6   7

filen

n1  n2  n3  n4
nn  nm  no  np

my desired output is to produce 4 data files by collecting respective columns from the datafile.

Output1

file1   file2   file3   filen
a   c   11  n1
1   g   2   nn

Output2

file1   file2   file3   filen
a   d   12  n2
2   h   4   nm

Output3

file1   file2   file3   filen
b   e   23  n3
3   i   6   no

Output4

file1   file2   file3   filen
b   f   12  n4
4   g   7   np

I was able to copy and organize in to the four desired files using

awk 'FNR==1{f++}{a[f,FNR]=$1}END{for(x=1;x<=FNR;x++){for(y=1;y<ARGC;y++)printf("%s ",a[y,x]);print ""}}' file* > output1
awk 'FNR==1{f++}{a[f,FNR]=$2}END{for(x=1;x<=FNR;x++){for(y=1;y<ARGC;y++)printf("%s ",a[y,x]);print ""}}' file* > output2
awk 'FNR==1{f++}{a[f,FNR]=$3}END{for(x=1;x<=FNR;x++){for(y=1;y<ARGC;y++)printf("%s ",a[y,x]);print ""}}' file* > output3
awk 'FNR==1{f++}{a[f,FNR]=$4}END{for(x=1;x<=FNR;x++){for(y=1;y<ARGC;y++)printf("%s ",a[y,x]);print ""}}' file* > output4

However, I couldn't append the file name in the first row of the output files.

Any help would be appreciated

5
  • 1
    Do the output files already exist? Why are you using them in your paste command?
    – terdon
    Feb 19, 2016 at 23:17
  • I think he meant to put his input files there and then produce output1 with all fist columns, output2 all second columns and so on. The command is just mixed up
    – FelixJN
    Feb 19, 2016 at 23:23
  • Yes, the output files do exist, I wanted to add a raw which is the file name where the columns are copied. The output file is going to have n+1 rows than the given n raw files. The idea is to add the variable name in each column.
    – AiB
    Feb 19, 2016 at 23:25
  • Please edit your question and clarify. If you already have the Output files, why mention the input files at all? We need to know what you have and what you want to get.
    – terdon
    Feb 19, 2016 at 23:28
  • @terdon, sorry for the confusion I created. I have corrected my question. I meant I have the results using the above code.
    – AiB
    Feb 19, 2016 at 23:41

3 Answers 3

1

If you already have the output files and just need to add a header with the file names to each of them, all you need is something like:

header=$(printf "%s %s %s %s\n" $(awk 'FNR==1{print FILENAME}' file*))
for file in output*; do 
    printf '%s\n%s\n' "$header" "$(cat $file)" > tmp && 
    mv tmp "$file"; 
done
2
  • Thanks terdon. it runs and works in this case. I am curios however, if I run the two codes separately, the order may not be kept which is going to very bad if it happens. Plus if I have more than four input data file, how "%s %s %s %s\n" will be changed so that it will work for n data files?
    – AiB
    Feb 20, 2016 at 0:22
  • @Abraham I'm not sure what you mean about the order, but to get more names, you could do something like header=$(awk 'FNR==1{printf "%s ", FILENAME}' * |sed 's/ $/\n/').
    – terdon
    Feb 20, 2016 at 13:39
0

Simply echo the file and then use the awk-processed data:

paste <( echo file1 ; awk '{print $1}' file1 )\
      <( echo file2 ; awk '{print $1}' file2 ) #and so on

The commands in parentheses get executed in their own subshells before being passed as input to paste

1
  • Even simpler. I should really get into awk more....
    – FelixJN
    Feb 19, 2016 at 23:24
0

I would do the whole thing like this:
Use printf + cut to print the header, paste the files content and concatenate the two results then pipe that to awk and print every fourth column starting with the 1st one to output1, every fourth column starting with the 2nd one to output2 and so on:

for f in file*
do
printf " ${f}%.0s" 1 2 3 4
done | cut -c2- | cat - <(paste file*) | awk '{
for (i=1;i<=NF;i+=4){printf "%s ",$i >"output1"} ;print "" >"output1"
for (i=2;i<=NF;i+=4){printf "%s ",$i >"output2"} ;print "" >"output2"
for (i=3;i<=NF;i+=4){printf "%s ",$i >"output3"} ;print "" >"output3"
for (i=4;i<=NF;i+=4){printf "%s ",$i >"output4"} ;print "" >"output4"
}'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.