Comma-separated values: files containing data arrange in a table, with commas to separate columns

learn more… | top users | synonyms

2
votes
4answers
364 views

Concatenate filename inside of CSV file for each line for multiple CSV files

I have lots of CSV files and I need the filename of each specific CSV file in every line of each file. Original file content of abc123.csv: ColVal_1;ColVal_2;ColVal_3 ColVal_4;ColVal_5;ColVal_6 ...
0
votes
3answers
52 views

Group by and sum in shell script without awk

I have a file like: $ cat input.csv 201,100 201,300 300,100 300,500 100,400 I want to add the values in column 2 which has same value in column 1. Expected output is as follows: $ cat output.csv ...
0
votes
1answer
990 views

Selecting rows in a CSV file based on column value

I have a CSV file with 4 columns: Itemname,Value,Description and component which is quite huge. I have to generate a template from the above CSV file that displays only the rows of the specified ...
3
votes
0answers
336 views

Catdoc xls2csv converting only one sheet

I have Catdoc's xls2csv installed on Debian/Squeeze. It used to work beautifully when I copied files to a folder like /var/www/xyz, converting all sheets into .csv format, separated by ^L. But now, ...
0
votes
1answer
57 views

How can I convert this excel file so that it is not only one row?

I've got an Excel file, shown in the picture below, and available for download here. What I need is to extract the variables under Item (Column B) and the values in column G. As a start, I tried ...
0
votes
2answers
26 views

Grep matching cell of the csv file and index of that row

I have csv file and I want to see numbers that start with 0.99 and their row's index which is the first cell of that row. This is what I have so far: cat fil.csv | grep '0\\.99'| tee > (cut -d, ...
1
vote
2answers
43 views

find field pattern in a csv file then count unique field values

I am using Cygwin (bash) to create a script to find, group and count fields in multiple CSV files. Each row will have comma-separated fields with each field following a similar convention. There is a ...
0
votes
1answer
39 views

insert comma after the last letter

I use a txt file, and I need to convert it in csv file Saint Petersburg 0 10 0.1 - N Moscow - 9 0 - N Novgorod 0 7 1 30 Y In ...
2
votes
6answers
709 views

Handling comma in string values in a CSV file [duplicate]

I have a comma-separated file which has numeric and string columns. String columns are quoted and can have comma in between the quotes. How do I identify the columns with FS =","? Sample records ...
2
votes
2answers
646 views

how to convert multiple txt to CSV with field data separated by blank lines

I have some data in multiple text files where fields are separated by blank lines. There are only 4 fields but in the second field there are more subfields, could be three or more. The first field is ...
0
votes
1answer
38 views

export from MYSQL to CSV

How do I export from MYSQL to CSV in mariadb on Centos 7? I am using: Server version 5.5.37-MariaDB Protocol version 10 Connection Localhost via UNIX socket UNIX socket ...
2
votes
1answer
56 views

splitting a CSV and keeping the header without intermediate files

I am trying to split a dozen 100MB+ csv files into managable smaller files for a curl post. I have managed to do it but with a lot of temporary files and IO. It's taking an eternity. I am hoping ...
1
vote
1answer
98 views

Confused about how to convert XML to CSV using xmlstarlet on OS X?

I asked a question earlier and it was closed as being a duplicate: How to parse XML to CSV with a shell script?. The answer in the duplicate that was linked suggested using a tool called XMLStarlet, ...
5
votes
4answers
173 views

Using jq to extract values and format in CSV

I have the below json file: { "data": [ { "displayName": "First Name", "rank": 1, "value": "VALUE" }, { "displayName": "Last Name", "rank": 2, ...
0
votes
1answer
81 views

Save file as Tab delimited text and spreadsheet

I am using the following command line in Linux to save the content of my input file (txt file consisted of columns) as a spreadsheet: less input_file > out_put.csv my output file is The ...
0
votes
1answer
67 views

Cut a specific part of a column from a csv file [closed]

I have to process a csv containing about 15 columns one of which contains a particular timestamp that includes milliseconds. When I go to insert this csv into a DB, I get an error because of this ...
1
vote
2answers
532 views

Shell script to Capture the file name and size in csv file and add header on that file

I have a number of zip files in a certain folder. I want to capture file name and the size of the files in a CSV format in day wise and put it in crontab so I can report on a daily basis, and then ...
2
votes
3answers
2k views

Select lines from text file which have ids listed in another file

I use a lot of grep awk sort in my unix shell to work with medium-sized (around 10M-100M lines) tab-separated column text files. In this respect unix shell is my spreadsheet. But I have one huge ...
18
votes
11answers
7k views

Is there a robust command line tool for processing csv files?

I work with CSV files and sometimes need to quickly check the contents of a row or column from the command line. In many cases cut, head, tail, and friends will do the job; however, cut cannot easily ...
-1
votes
1answer
72 views

search for a character in one column and replace value in another column of matched records

I have a requirement to find value 2 in column 4 and wherever it matches replace 6th column with '-'sign in the beginning suppose if i have csv file - file1 like this abc,0,def,1234,ijk ...
8
votes
2answers
287 views

Reindexing a large CSV file

I went through the answers in this helpful thread, but my problem seems to be different enough that I can't think of good answer (at least with sed). I have a large CSV file (200+ GB) with rows that ...
-1
votes
3answers
153 views

How can I select columns based on a control file with invalid column numbers?

I have to generate the file dynamically from the source file based on the below control file. control_file.txt 1,3,5,-1,8,-1,4 The control file contains the position of columns which I require ...
1
vote
2answers
81 views

Count the maximum character length for all the data fields in a csv file and output to txt

Given a CSV with many data fields (>50), how can I count the maximum character length for each data field and then export all the counts to a txt file? BTW, I want to ignore the first line of the file ...
3
votes
3answers
260 views

Determine maximum column length for every column in csv-file

To determine the maximum length of each column in a comma-separated csv-file I hacked together a bash-script. When I ran it on a linux system it produced the correct output, but I need it to run on OS ...
2
votes
2answers
151 views

Delete rows in a csv file based on a specific column value

I have a csv file with the following headers - Interaction_ID, PDB_ID, First_Residue, First_Chain, Second_Residue, Third_Residue, Third_Chain & Pattern: ...
4
votes
2answers
912 views

How to display TSV (csv) in console, when empty cells are missed by: `column -t -s $'\t' `

I have file with columns spearated with tab. I have file when some rows have empty cells (on begining, in middle). In such cases column -t -s $'\t' simply fails: Input: $ echo -e ...
8
votes
4answers
3k views

command to layout tab separated list nicely

Sometimes, I'm getting as an input tab separated list, which is not quite aligned, for instance var1 var2 var3 var_with_long_name_which_ruins_alignment var2 var3 Is there an easy way to render ...
4
votes
5answers
7k views

extract the 4th column from a csv file using unix command

I have a csv file formatted as below. "col1","col2","col3","col4" "1","text1","<p>big html text</p> ","4th column" "2","text2","<p>big2 html2 text2</p> ...
-2
votes
2answers
206 views

Extract certain column from CSV files [closed]

I have a directory with a group of CSV files. All files have the same column headers. I need to extract values in a certain column from all files. The common part in files names is ...
0
votes
3answers
76 views

Remove all capital letters from a particular column in a CSV file

I am having a CSV file containing 3000+ lines in the following format: R1C1, ABC123abc, R1C3 R2C1, DEFG456defg, R2C3 R3C1, HIJKL789hijkl, R3C3 I need to remove all the capital letters from the ...
4
votes
4answers
327 views

Convert key=value blocks to CSV

I am trying to transpose a content of a file into another. I am new to shell scripting any help would be really appreciated Input file Test.txt: HLRSN = 3 IMSI = 404212109727229 KIVALUE = ...
3
votes
1answer
286 views

How to export data in the terminal to a file (e.g. csv file)

I am a complete beginner in using Linux and I would like to know how to export a bunch of data in the command line to a csv or spreadsheet file (or even text would work). Right now I am simply using ...
1
vote
0answers
310 views

How to create CSV file without unnecessary spaces

I use the xls2csv binary in order to translate XLS doc to CSV in my linux red-hat machine, example: ( from man page ) xls2csv -x "1252spreadsheet.xls" -b WINDOWS-1252 -c "ut8csvfile.csv" -a UTF-8 ...
-3
votes
2answers
235 views

Shell Script to capture file name and file size in CSV format [closed]

I have number of zip files in a certain folder. I want to capture file name and the size of the file in a csv format containing file names and sizes, one file per day. The file names do contain the ...
2
votes
2answers
337 views

CSV file + set CSV parameters and values in bash script

I have the following CSV file: Alabama,Alaska,Arizona,Arkansas,California,Colorado,Connecticut,Delaware,Florida,Georgia,Hawaii,Idaho,Illinois,Indiana,Iowa 1000,"1 0 0 1",1002,1002,1003,1004,1005,"1 0 ...
1
vote
2answers
212 views

merge csv files, escape comma inside quotes

I have 3 csv files I want to join by first column (id column) Each file has the same 3 columns Row example : id | timestamp | Name 3792318, 2014-07-15 00:00:00, "A, B" When I join ...
1
vote
5answers
463 views

delete lines in a csv file older than 7 days

I have a csv that I need to remove all lines that are older than 7 days. This is the format of the csv Person ID VIP CS SS LT FTLT PS Modified Datestamp T001028 1 1 1 0 0 0 ...
3
votes
1answer
439 views

merge csv files by first column

I have 3 csv files like this. csv 1: 1,aaaa,bbb,2014-04-01 2,qwe,rty,2014-04-03 3,zxc,cvb,2014-04-05 csv 2: 2,j,k,2014-04-01 3,a,s,2014-04-04 5,g,h,2014-04-08 csv 3: 2,a,s,d,f,g,2014-04-01 ...
1
vote
2answers
195 views

Merge csv files with conditions

I have 2 CSV files for each date (csv_2014_4_15 and csv_2014_4_16) , with basic structure and couple of unique columns as below. id,name,created_at,updated_at,other columns 12, joe, 2013-1-1 18:30, ...
11
votes
5answers
11k views

Remove comma between the quotes only in a comma delimited file

I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row 123,"ABC, DEV 23",345,534.202,NAME I need to ...
0
votes
3answers
51 views

how to export all numbers that are unique in few text files into another file?

I have a.txt,b.txt,c.txt. Each has different numbers as below: a.txt: 12 14 111 1 15 2 b.txt 12 18 22 23 1 2 c.txt 12 14 15 16 17 1200 The output should contains all the numbers from ...
0
votes
2answers
38 views

Trim last chars from a line

I have a csv file and the lines have different length. But all of them ends with a number surrounded by a pair of braces. For example, ........{5} ........{16} I'd like to remove the number and the ...
2
votes
3answers
798 views

replace column of a csv file with column from another file

I have two sample files like this: $ cat file1 abc,sachin cat,kumar $ cat file2 xyz pressure $ cat file3 xyz,sachin pressure, kumar I want the first column of file1 to be replaced with file2. ...
5
votes
2answers
2k views

Is there a command line utility to transpose a csv-file?

Given a file like so First, Last, Age Cory, Klein, 27 John Jacob, Smith, 30 Is there a command line utility to transpose the contents so the output appears like so First, Cory, John Jacob Last, ...
6
votes
3answers
903 views

Remove only the commas present within the double quotes

In a text file, I want to remove , (commas) and also the " (quotes) (only if the double quotes contains numbers seperated by commas). 56,72,"12,34,54",x,y,"foo,a,b,bar" Expected ouput ...
11
votes
8answers
1k views

How do I get the character count of words in a particular column?

I have a CSV file like this: abd,123,egypt,78 cde,456,england,45 How can I get the character count of only the 3rd column words? I can't figure out how to get wc to do this.
17
votes
4answers
37k views

Convert a .xlsx (MS Excel) file to .csv on command line with semicolon separated fields

I realize that this is not an entirely unix/linux related question. But since this is something I'll do on linux, I hope someone has an answer. I have an online excel file (.xlsx) which gets updated ...
2
votes
3answers
3k views

Replacing the values in one file with the values in another file in bash

I am having a csv file naming List.csv in following format: Location,IP Address,Host Name,Domain,Domain Name, User Name,Manufacturer,Model,System Type, Serial Number, Operating System,RAM ...
5
votes
3answers
4k views

Filter a .CSV file based on the 5th column values of a file and print those records into a new file

I have a .CSV file with the below format: "column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10 "12310","42324564756","a simple string with a , ...
3
votes
1answer
50 views

update line based on previous line

I have a issue since I'm not familiar with awk. I have csv file generated from sar -d output converted to csv style: 12:33:41 unix,restarts 12:35:00,lofi4096,0,0.0,0,0,0.0,0.0 ...