Comma-separated values: files containing data arrange in a table, with commas to separate columns

learn more… | top users | synonyms

0
votes
2answers
172 views

Extract given column from comma-separated values with inner commas and quotation marks [duplicate]

I have a file in which the fields are separated by comma. Some example input: col1,"1,2",col3 col1,"1,2,3",col3 col1,"1 2,3",col3 col1,"1 "2,3"",col3 Now, I have to fetch the second column, so ...
1
vote
5answers
394 views

delete lines in a csv file older than 7 days

I have a csv that I need to remove all lines that are older than 7 days. This is the format of the csv Person ID VIP CS SS LT FTLT PS Modified Datestamp T001028 1 1 1 0 0 0 ...
-1
votes
1answer
712 views

Selecting rows in a CSV file based on column value

I have a CSV file with 4 columns: Itemname,Value,Description and component which is quite huge. I have to generate a template from the above CSV file that displays only the rows of the specified ...
2
votes
1answer
196 views

Parse csv using awk [duplicate]

Possible Duplicate: Is there a robust command line tool for processing csv files? I am trying to parse the second line in a file (which contains commas within double quotes) "100 Watt ...
3
votes
0answers
278 views

Catdoc xls2csv converting only one sheet

I have Catdoc's xls2csv installed on Debian/Squeeze. It used to work beautifully when I copied files to a folder like /var/www/xyz, converting all sheets into .csv format, separated by ^L. But now, ...
2
votes
2answers
76 views

How can I show the number of positions lines have moved in diff output?

top1 1,facebook.com 2,google.com 3,youtube.com 4,yahoo.com 5,wikipedia.org 6,baidu.com 7,live.com 8,amazon.com 9,qq.com 10,twitter.com top2 1,facebook.com 2,google.com 3,youtube.com 4,yahoo.com ...
2
votes
2answers
1k views

Data pivot with awk

I'm trying to pivot a file using awk. This is an example of my input File: VarName;TimeString;VarValue;Validity;Time_ms A;23.11.201215:03:53;1;1;41236627696,7593 ...
4
votes
2answers
2k views

Is there a command line utility to transpose a csv-file?

Given a file like so First, Last, Age Cory, Klein, 27 John Jacob, Smith, 30 Is there a command line utility to transpose the contents so the output appears like so First, Cory, John Jacob Last, ...
1
vote
4answers
1k views

How to rename files with sed and csv

I'm new with Linux, sed, and awk but I don't mind challenging myself on new ideas. That being said, I understand the purpose and how to use rename and sed for a common event such as adding a $date or ...
4
votes
4answers
1k views

A unix command to truncate each line of a file

I have a CSV file and I want to truncate it from the third semicolon. For example, if I have this file: 1;foo;bar;baz;x;y;z 2;foo;bar;baz;x;y;z 3;foo;bar;baz;x;y;z I want to get the following ...
2
votes
3answers
4k views

Extracting column from comma separated text

I have a long comma-separated delimited file with 20K lines. Here's a sample: "","id","number1","number2","number3","number4","number5","number6","number7" ...
8
votes
5answers
9k views

Remove comma between the quotes only in a comma delimited file

I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row 123,"ABC, DEV 23",345,534.202,NAME I need to ...
2
votes
1answer
125 views

Read only new parts of a file?

I have a large CSV file, that grows by some entries every day (created by an external software which writes always to this one file, no splitting possible). For performance reasons and further ...
2
votes
1answer
564 views

Regarding generating intersection and union of two csv files

I have two csv files, there are some overlap columns between these two files. Assume one file is called as A.csv and another is called as B.csv. The intersection of A and B is called as C. I would ...
3
votes
3answers
607 views

Using CSV line as command parameters

I have a CSV file like: Name,Age,Address Daniel Dvorkin,28,Some Address St. 1234 (... N ...) Foo Bar,90,Other Address Av. 3210 And I have a command that take this parameters: ./mycommand ...
2
votes
3answers
585 views

How can I turn ugly output into pretty, useful information?

How can I turn this ugly output into pretty, useful data? The output: /* ---------- TA#box#AbC_p ---------- */ insert_job: TA#box#AbC_p job_type: a #owner: bob permission: gx date_conditions: 1 ...
2
votes
1answer
2k views

Swap two columns in a CSV using SED

I have a CSV file that contains 10 different fields (, is the deliminator). Example data: student-id,last,first,hwk1,hwk2,hwk3,exam1,hwk4,hwk5,exam2 pts-avail,,,100,150,100,200,150,100,300 ...
5
votes
3answers
5k views

Search for values in 2 columns from CSV

I have a csv files with say 20 odd columns. I need to get the 14th and 15th column values where value in 14th column is say "VALUE1" and value in 15th column is "VALUE2". My condition gets satisfied ...
2
votes
6answers
4k views

How to swap columns in such a file?

I have a text file, each line is stored like this : "Video or movie" "parent" "Media or entertainment" "1" "1" "1" "0" "0" I want to swap the columns 3 with 2, i.e. "Video or movie" ...
3
votes
2answers
368 views

need shell script to grock a csv

Need some pointing in right direction on script to fetch and regex or sed. Site24x7 provides a URL with a CSV list of their source IP's used for monitoring. (they also provide other formats, CSV ...
10
votes
2answers
2k views

sort inplace - like sed --in-place - exists?

Am I blind or is there no option like --in-place for sort? In order to save results to the input file, sed uses -i (--in-place). Redirecting the output of sort to the input file sort < f > f ...
3
votes
1answer
121 views

Wrapping long cells in a tsv to keep them in same column

Formatting a TSV with long cells with column makes its readability very low. It simply breaks lines like this: $ python3 -c 'for i in (1,2,3): print(((str(i)*50)+"\t")*3)' | column -s $'\t' -t ...
4
votes
2answers
732 views

How to display TSV (csv) in console, when empty cells are missed by: `column -t -s $'\t' `

I have file with columns spearated with tab. I have file when some rows have empty cells (on begining, in middle). In such cases column -t -s $'\t' simply fails: Input: $ echo -e ...
4
votes
5answers
4k views

How can I convert tab delimited data to comma delimited data?

I'm requesting a list of ec2 snapshots via amazon's ec2 command line tool: ec2-describe-snapshots -H --hide-tags > snapshots.csv The data looks something like this: SnapshotId VolumeId ...
14
votes
4answers
19k views

How to manipulate a CSV file with sed or awk?

How can I do the following to a CSV file using sed or awk? Delete a column Duplicate a column Move a column I have a big table with over 200 rows, and I'm not that familiar with sed.
2
votes
1answer
250 views

Extract pitch information from audio file

Is there a command-line utility which would allow me to extract pitch information from an audio file and store it in a numerical form (i.e. as a list of comma-separated values)?
3
votes
1answer
371 views

An efficient filter for .sql files with just “INSERT INTO”

Several times I've found myself working on data that has been exported to .sql files that I want to process rather than import into a database. I'm only talking about "pure data" sql files. Ones that ...
17
votes
5answers
4k views

Command line friendly spreadsheets

Does such a thing exist? Text-based spreadsheets that display well in a CLI environment. I'm aware that I could cat foobar.csvand do as I please, but it isn't particularly practical or attractive. I ...
17
votes
4answers
33k views

Convert a .xlsx (MS Excel) file to .csv on command line with semicolon separated fields

I realize that this is not an entirely unix/linux related question. But since this is something I'll do on linux, I hope someone has an answer. I have an online excel file (.xlsx) which gets updated ...
2
votes
4answers
959 views

gawk join two TSV by columns (a'la sql join)

How to join, to tsv files, examples: a.tsv c 7 r z d 6 s w f 1 f f b 8 p y a 9 q x b.tsv a q a c r ccc b p bb 0 0 0 d s dddd Here I'd like to ...
6
votes
3answers
844 views

Use `less` to view comma-aligned data

I have a big CSV file. I would like to view my file using less or some command like it which doesn't have to read the whole file at once to show me part of it. Is there a command out there which can ...
1
vote
1answer
249 views

Need help processing a text file with awk to conform to CSV flat file format

I have the following problem. I collected data on reaction time from over 100 participants for an experiment I am running. Unfortunately, the separators between fields were not consistent, but after a ...
2
votes
4answers
6k views

How can I find and replace with a new line?

I have a CSV delimited by commas and I want to delimit it by newlines instead. Input: a, b, c Output: a b c I've written Java parsers that do this stuff, but couldn't this be done with vim or ...
8
votes
4answers
3k views

command to layout tab separated list nicely

Sometimes, I'm getting as an input tab separated list, which is not quite aligned, for instance var1 var2 var3 var_with_long_name_which_ruins_alignment var2 var3 Is there an easy way to render ...
18
votes
10answers
6k views

Is there a robust command line tool for processing csv files?

I work with CSV files and sometimes need to quickly check the contents of a row or column from the command line. In many cases cut, head, tail, and friends will do the job; however, cut cannot easily ...