0

Say the directory contained 3 csv files:

The first csv:

Name, John
Age, 18

The second csv:

Name, Jim
Age, 21

The third csv:

Name, Amy
Age, 22

I would want the result to be:

Name, John, Jim, Amy
Age, 18, 21, 22

It's important to know the directory could have n many csvs I have both bash and posix shell available

Edit:

This feels like it should work but still has an issue with regards to order:

awk -F, -v OFS="," '{a[FNR]=a[FNR]?a[FNR]FS$2:$1FS$2}END{for(x in a)print x,a[x]}' *.csv > results.csv

Which makes no sense as FNR 1 should be first in the array but it is printed last?

4
  • No unfortunately that post uses join which works on two files rather than working over a directory, I could of course do it in a loop but I feel like theres probably a simple one liner with awk
    – Tap
    Apr 20 at 19:39
  • 2
    Okay so I got pretty close: awk -F, -v OFS="," '{a[$1]=a[$1]?a[$1]FS$2:$2}END{for(x in a)print x,a[x]}' *.csv > results.csv
    – Tap
    Apr 21 at 11:30
  • 2
    But it doesn't preserve the row order, for example Age is now at the top as its sorted in the associative array alphabetically
    – Tap
    Apr 21 at 11:33
  • Well done for your effort. I wrote recently a very similar answer, the main modification is that we keep the first file as is, so it was easier for me to provide some help here.
    – thanasisp
    Apr 21 at 12:57

1 Answer 1

1

Your attempt was very close. Here are some modifications to make it work as required:

awk -F, -v OFS="," '{
        a[FNR] = (FNR==NR ? $0 : a[FNR] OFS $2)
    }
    END { 
        for(i=1;i<=FNR;i++) print a[i]
    }' *.csv

for (x in a) does not guarantee the array access order, so in order to keep the order, we run a numeric index into the for loop. We could also use for (i=1;i<=length(a);i++), but it's good to know that passing an array as an argument to length() is not standard for all awks, you can see this into the GNU awk manual.

See also: Scanning an Array

Also we use the conditional FNR==NR which is an idiom for "when parsing the first file", and want to save the whole line $0, for the next files we just append the second field.

2
  • Thats cleaner thank you, whats the logic behind using OFS over FS?
    – Tap
    Apr 21 at 14:58
  • It's the same here, as both are the comma, so we could use FS also. In general, we could need to output a different separator than the input one, so, as a habit, it's good to write OFS there (usually I 've been writing FS:).
    – thanasisp
    Apr 21 at 15:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.