This question already has an answer here:

Consider a large number of CSV files (*.csv) living in some folder. They all have the same exact header.

How can I efficiently concatenate them all into a single CSV file with the same single header?

I found a number of solutions that solve similar but more specific problems.

The current awk solution doesn't work.

$ cat
    #!/usr/bin/env zsh
    awk '
        FNR==1 && NR!=1 { while (/^<header>/) getline; }
        1 {print}
    ' $1/*.csv > $2

$ ./concat_my_csv_files /some/path/to/csv/files/ full_join.csv

when I do:

grep -F column_A full_join.csv

I see several rows having it.

marked as duplicate by Ramesh, roaima, hildred, cas, Raphael Ahrens Oct 6 '15 at 4:35

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

up vote 3 down vote accepted
awk '
    NR == 1 {print}
    FNR == 1 {next}
' *.csv

The NR variable is the record number of all the input.
The FNR variable is the record number of only the current file.

This prints the first line seen by awk (the header of the first file), then will skip the first line of each file, printing all the other lines.

  • Note, this can be logically shortened to: awk 'NR == 1 || FNR > 1' *.csv – glenn jackman Oct 5 '15 at 21:25
  • How are we supposed to help you when you don't show us what your files look like? In the absence of actual information, I'm assuming your "header" is one line at the top of each file. – glenn jackman Oct 6 '15 at 1:07

Basically you want "head -n 1 firstorany.csv; tail -n +2 *.csv".

set -- *.csv
head -n 1 "$1"
tail -n +2 "$@"

If you have *.csv as arguments in a sh script, omit the first line.

  • Thanks. How do I control where the output is written here? (e.g. see in the OP update). – Amelio Vazquez-Reina Oct 5 '15 at 22:33
  • 1
    ( head ...; tail ... ) > output.csv – Christoph Berg Oct 7 '15 at 14:44

Not the answer you're looking for? Browse other questions tagged or ask your own question.