Files containing data arrange in a table, often with commas (hence Comma Separated Values), to separate columns. Rows are separated by newlines (but not all newlines are row separators as fields can be quoted to contain the separator newlines. Use this tag for full-fledged CSV data not the simpler ...

learn more… | top users | synonyms

3
votes
4answers
2k views

How to rename files with sed and csv

I'm new with Linux, sed, and awk but I don't mind challenging myself on new ideas. That being said, I understand the purpose and how to use rename and sed for a common event such as adding a $date or ...
5
votes
4answers
2k views

A unix command to truncate each line of a file

I have a CSV file and I want to truncate it from the third semicolon. For example, if I have this file: 1;foo;bar;baz;x;y;z 2;foo;bar;baz;x;y;z 3;foo;bar;baz;x;y;z I want to get the following ...
2
votes
4answers
6k views

Extracting column from comma separated text

I have a long comma-separated delimited file with 20K lines. Here's a sample: "","id","number1","number2","number3","number4","number5","number6","number7" ...
12
votes
5answers
16k views

Remove comma between the quotes only in a comma delimited file

I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row 123,"ABC, DEV 23",345,534.202,NAME I need to ...
2
votes
1answer
152 views

Read only new parts of a file?

I have a large CSV file, that grows by some entries every day (created by an external software which writes always to this one file, no splitting possible). For performance reasons and further ...
2
votes
1answer
837 views

Regarding generating intersection and union of two csv files

I have two csv files, there are some overlap columns between these two files. Assume one file is called as A.csv and another is called as B.csv. The intersection of A and B is called as C. I would ...
4
votes
3answers
968 views

Using CSV line as command parameters

I have a CSV file like: Name,Age,Address Daniel Dvorkin,28,Some Address St. 1234 (... N ...) Foo Bar,90,Other Address Av. 3210 And I have a command that take this parameters: ./mycommand ...
2
votes
3answers
738 views

How can I turn ugly output into pretty, useful information?

How can I turn this ugly output into pretty, useful data? The output: /* ---------- TA#box#AbC_p ---------- */ insert_job: TA#box#AbC_p job_type: a #owner: bob permission: gx date_conditions: 1 ...
3
votes
2answers
3k views

Swap two columns in a CSV using SED

I have a CSV file that contains 10 different fields (, is the deliminator). Example data: student-id,last,first,hwk1,hwk2,hwk3,exam1,hwk4,hwk5,exam2 pts-avail,,,100,150,100,200,150,100,300 ...
5
votes
3answers
9k views

Search for values in 2 columns from CSV

I have a csv files with say 20 odd columns. I need to get the 14th and 15th column values where value in 14th column is say "VALUE1" and value in 15th column is "VALUE2". My condition gets satisfied ...
4
votes
6answers
6k views

How to swap columns in such a file?

I have a text file, each line is stored like this : "Video or movie" "parent" "Media or entertainment" "1" "1" "1" "0" "0" I want to swap the columns 3 with 2, i.e. "Video or movie" ...
3
votes
2answers
416 views

need shell script to grock a csv

Need some pointing in right direction on script to fetch and regex or sed. Site24x7 provides a URL with a CSV list of their source IP's used for monitoring. (they also provide other formats, CSV ...
22
votes
4answers
6k views

sort inplace - like sed --in-place - exists?

Am I blind or is there no option like --in-place for sort? In order to save results to the input file, sed uses -i (--in-place). Redirecting the output of sort to the input file sort < f > f ...
3
votes
1answer
128 views

Wrapping long cells in a tsv to keep them in same column

Formatting a TSV with long cells with column makes its readability very low. It simply breaks lines like this: $ python3 -c 'for i in (1,2,3): print(((str(i)*50)+"\t")*3)' | column -s $'\t' -t ...
6
votes
2answers
2k views

How to display TSV (csv) in console, when empty cells are missed by: `column -t -s $'\t' `

I have file with columns spearated with tab. I have file when some rows have empty cells (on begining, in middle). In such cases column -t -s $'\t' simply fails: Input: $ echo -e ...
5
votes
5answers
7k views

How can I convert tab delimited data to comma delimited data?

I'm requesting a list of ec2 snapshots via amazon's ec2 command line tool: ec2-describe-snapshots -H --hide-tags > snapshots.csv The data looks something like this: SnapshotId VolumeId ...
18
votes
5answers
29k views

How to manipulate a CSV file with sed or awk?

How can I do the following to a CSV file using sed or awk? Delete a column Duplicate a column Move a column I have a big table with over 200 rows, and I'm not that familiar with sed.
2
votes
1answer
295 views

Extract pitch information from audio file

Is there a command-line utility which would allow me to extract pitch information from an audio file and store it in a numerical form (i.e. as a list of comma-separated values)?
3
votes
1answer
480 views

An efficient filter for .sql files with just “INSERT INTO”

Several times I've found myself working on data that has been exported to .sql files that I want to process rather than import into a database. I'm only talking about "pure data" sql files. Ones that ...
21
votes
5answers
5k views

Command line friendly spreadsheets

Does such a thing exist? Text-based spreadsheets that display well in a CLI environment. I'm aware that I could cat foobar.csvand do as I please, but it isn't particularly practical or attractive. I ...
17
votes
4answers
51k views

Convert a .xlsx (MS Excel) file to .csv on command line with semicolon separated fields

I realize that this is not an entirely unix/linux related question. But since this is something I'll do on linux, I hope someone has an answer. I have an online excel file (.xlsx) which gets updated ...
2
votes
4answers
1k views

gawk join two TSV by columns (a'la sql join)

How to join, to tsv files, examples: a.tsv c 7 r z d 6 s w f 1 f f b 8 p y a 9 q x b.tsv a q a c r ccc b p bb 0 0 0 d s dddd Here I'd like to ...
6
votes
3answers
1k views

Use `less` to view comma-aligned data

I have a big CSV file. I would like to view my file using less or some command like it which doesn't have to read the whole file at once to show me part of it. Is there a command out there which can ...
1
vote
1answer
392 views

Need help processing a text file with awk to conform to CSV flat file format

I have the following problem. I collected data on reaction time from over 100 participants for an experiment I am running. Unfortunately, the separators between fields were not consistent, but after a ...
3
votes
4answers
13k views

How can I find and replace with a new line?

I have a CSV delimited by commas and I want to delimit it by newlines instead. Input: a, b, c Output: a b c I've written Java parsers that do this stuff, but couldn't this be done with vim or ...
13
votes
4answers
5k views

command to layout tab separated list nicely

Sometimes, I'm getting as an input tab separated list, which is not quite aligned, for instance var1 var2 var3 var_with_long_name_which_ruins_alignment var2 var3 Is there an easy way to render ...
23
votes
11answers
11k views

Is there a robust command line tool for processing csv files?

I work with CSV files and sometimes need to quickly check the contents of a row or column from the command line. In many cases cut, head, tail, and friends will do the job; however, cut cannot easily ...