3

I'm currently working with some CSV datasets and was wondering whether there are any good (even better if standard) CLI ways of manipulating CSV data.

To be more precise, I mainly want to be able to:

  • vertically concatenate CSVs
  • SQL-style join CSVs on one or more columns
1
  • @Kusalananda, though see info gawk fpat for some way for gawk to handle a little more complex csvs. Sep 12, 2017 at 9:57

1 Answer 1

11

Yes: CSVkit. http://csvkit.readthedocs.io/

CSV is not a standard that has anything to do with Unix, hence there are no "standard" (as in POSIX) utility for working with CSV files.

To vertically concatenate CSV files, use something like paste -d ',' file1.csv file2.csv (unless the CSV data contains newlines). CSVKit will handle most other kinds of operations (including queries and database loading, re-formatting etc.) on CSV files.

CSVKit is a collection of Python utilities. Among these is csvjoin, for example.

6
  • Note that ksh93 has built-in support to process csvs (not all flavours though). Sep 12, 2017 at 9:55
  • What about vertically concatenating columns within the same file?
    – ptrcao
    Dec 21, 2019 at 9:10
  • 1
    @ptrcao That sounds like something you could do with a combination of cut and paste, but feel free to ask a new question about this.
    – Kusalananda
    Dec 21, 2019 at 9:13
  • cut and paste are just shell commands or a part of CSVkit?
    – ptrcao
    Dec 21, 2019 at 9:21
  • 1
    @ptrcao These are standard shell tools. As I don't know what your data is, I suggested using these. There is also a csvcut tool in CSVkit, and csvstack, but again I know nothing about your data and what it is you want to do. I suggest that you ask a brand new question about this rather than using the comment thread of other questions.
    – Kusalananda
    Dec 21, 2019 at 9:43

Not the answer you're looking for? Browse other questions tagged .