Comma-separated values: files containing data arrange in a table, with commas to separate columns

learn more… | top users | synonyms

-3
votes
1answer
32 views

Empty columns which were seen before

I have a large CSV file and want to empty certain columns if they were seen before. So I have (to illustrate my problem): Category | Subcategory ---------+------------ foo | bar foo | bar ...
-4
votes
1answer
58 views

Merge two csvfiles by column header

I have two csv files: success.csv "ID","Legacy ID","Field1", "Field2" 1,1111,google,news 2,2222,yahoo,news error.csv "Legacy ID","Field1","Field2","Message" 3333,aol,news,"Failed to upload ...
0
votes
1answer
26 views

csv file with multiple columns to zenity

How could I output the contents of a .csv file to a Zenity list. The file has multiple rows and columns and the rows are no fixed numbers. I would also like it so that when the user double clicks, on ...
1
vote
2answers
155 views

How to concatenate a variable number of csv, removing their header rows?

I have a directory with several hundred csv files whose filenames start with two digits {01..84}. Several hundred >> 84, so obviously some filenames start with the same prefix. I wish to concatenate ...
0
votes
0answers
25 views

How can I simply monitoring bandwidth

I have low-end servers for download-upload something. I need to monitoring bandwidth. I know some software for this, but I can't use heavy software on my servers. So that's why I have an idea: using ...
2
votes
4answers
1k views

How to rename files with sed and csv

I'm new with Linux, sed, and awk but I don't mind challenging myself on new ideas. That being said, I understand the purpose and how to use rename and sed for a common event such as adding a $date or ...
1
vote
2answers
26 views

change non-zero padded hour and minute into 24hr time

I have data that I need to change before running it through the pandas library in python. It is currently in a format that stores midnight values as 2400 and that should be changed to 0000. The ...
1
vote
2answers
52 views

Remove a line if regex exists only in 1st column of each line in a csv file

I have the following comma-delimited data (spaces used for visual clarity): 1 2014 23 2352 19.24 7.18 1 2014 23 2353 19.78 4.65 24 2014 23 2354 20.26 25.34 1 2014 23 2355 26.65 42.76 1 2014 23 ...
16
votes
5answers
24k views

How to manipulate a CSV file with sed or awk?

How can I do the following to a CSV file using sed or awk? Delete a column Duplicate a column Move a column I have a big table with over 200 rows, and I'm not that familiar with sed.
-1
votes
1answer
1k views

Selecting rows in a CSV file based on column value

I have a CSV file with 4 columns: Itemname,Value,Description and component which is quite huge. I have to generate a template from the above CSV file that displays only the rows of the specified ...
0
votes
0answers
16 views

Gzip Mutliple files using single command [duplicate]

Want to Gzip my all the .csv files into a gzip file. I am using this command. gzip *.csv; Problem File contains '//' in its name that's why command is giving error. Error : - '//File.csv' is not ...
4
votes
4answers
393 views

Convert key=value blocks to CSV

I am trying to transpose a content of a file into another. I am new to shell scripting any help would be really appreciated Input file Test.txt: HLRSN = 3 IMSI = 404212109727229 KIVALUE = ...
2
votes
2answers
847 views

how to convert multiple txt to CSV with field data separated by blank lines [closed]

I have some data in multiple text files where fields are separated by blank lines. There are only 4 fields but in the second field there are more subfields, could be three or more. The first field is ...
2
votes
2answers
157 views

Sorting a csv file [duplicate]

I need to sort a CSV file, but the the header row (1st row) keeps getting sorted. This is what I'm using: cat data1.csv | sort -t"|" -k 1 -o data1.csv Here's a sample line: ...
0
votes
1answer
75 views

Merging CSV files without losing the header line

I need to merge two CSV files: datasorted1.csv and datasorted2.csv, but my headers keep getting sorted. head -n 1 datasorted1.csv datasorted2.csv > resultado.csv && tail -n +2 ...
0
votes
2answers
155 views

Export file names of a directory to csv

I have a question regarding exporting folder names to a csv file. I have read already many entries in the web to such topics but on nearly all the topic was about exporting filenames and size and ...
0
votes
1answer
50 views

Is there a standalone tool which will write reports from CSV data files

I download CSV files (e.g. my bank account transactions) and would like to build reports (to print) from the data. I have searched and found several report writer libraries, but all would force me to ...
2
votes
4answers
538 views

Concatenate filename inside of CSV file for each line for multiple CSV files

I have lots of CSV files and I need the filename of each specific CSV file in every line of each file. Original file content of abc123.csv: ColVal_1;ColVal_2;ColVal_3 ColVal_4;ColVal_5;ColVal_6 ...
0
votes
3answers
113 views

Group by and sum in shell script without awk

I have a file like: $ cat input.csv 201,100 201,300 300,100 300,500 100,400 I want to add the values in column 2 which has same value in column 1. Expected output is as follows: $ cat output.csv ...
3
votes
0answers
360 views

Catdoc xls2csv converting only one sheet

I have Catdoc's xls2csv installed on Debian/Squeeze. It used to work beautifully when I copied files to a folder like /var/www/xyz, converting all sheets into .csv format, separated by ^L. But now, ...
0
votes
1answer
112 views

How can I convert this excel file so that it is not only one row?

I've got an Excel file, shown in the picture below, and available for download here. What I need is to extract the variables under Item (Column B) and the values in column G. As a start, I tried ...
0
votes
2answers
50 views

Grep matching cell of the csv file and index of that row

I have csv file and I want to see numbers that start with 0.99 and their row's index which is the first cell of that row. This is what I have so far: cat fil.csv | grep '0\\.99'| tee > (cut -d, ...
1
vote
2answers
102 views

find field pattern in a csv file then count unique field values

I am using Cygwin (bash) to create a script to find, group and count fields in multiple CSV files. Each row will have comma-separated fields with each field following a similar convention. There is a ...
0
votes
1answer
48 views

insert comma after the last letter

I use a txt file, and I need to convert it in csv file Saint Petersburg 0 10 0.1 - N Moscow - 9 0 - N Novgorod 0 7 1 30 Y In ...
2
votes
6answers
1k views

Handling comma in string values in a CSV file [duplicate]

I have a comma-separated file which has numeric and string columns. String columns are quoted and can have comma in between the quotes. How do I identify the columns with FS =","? Sample records ...
0
votes
1answer
114 views

export from MYSQL to CSV

How do I export from MYSQL to CSV in mariadb on Centos 7? I am using: Server version 5.5.37-MariaDB Protocol version 10 Connection Localhost via UNIX socket UNIX socket ...
2
votes
1answer
127 views

splitting a CSV and keeping the header without intermediate files

I am trying to split a dozen 100MB+ csv files into managable smaller files for a curl post. I have managed to do it but with a lot of temporary files and IO. It's taking an eternity. I am hoping ...
1
vote
1answer
366 views

Confused about how to convert XML to CSV using xmlstarlet on OS X?

I asked a question earlier and it was closed as being a duplicate: How to parse XML to CSV with a shell script?. The answer in the duplicate that was linked suggested using a tool called XMLStarlet, ...
5
votes
4answers
736 views

Using jq to extract values and format in CSV

I have the below json file: { "data": [ { "displayName": "First Name", "rank": 1, "value": "VALUE" }, { "displayName": "Last Name", "rank": 2, ...
0
votes
1answer
319 views

Save file as Tab delimited text and spreadsheet

I am using the following command line in Linux to save the content of my input file (txt file consisted of columns) as a spreadsheet: less input_file > out_put.csv my output file is The ...
0
votes
1answer
82 views

Cut a specific part of a column from a csv file [closed]

I have to process a csv containing about 15 columns one of which contains a particular timestamp that includes milliseconds. When I go to insert this csv into a DB, I get an error because of this ...
1
vote
2answers
816 views

Shell script to Capture the file name and size in csv file and add header on that file

I have a number of zip files in a certain folder. I want to capture file name and the size of the files in a CSV format in day wise and put it in crontab so I can report on a daily basis, and then ...
2
votes
3answers
2k views

Select lines from text file which have ids listed in another file

I use a lot of grep awk sort in my unix shell to work with medium-sized (around 10M-100M lines) tab-separated column text files. In this respect unix shell is my spreadsheet. But I have one huge ...
20
votes
11answers
8k views

Is there a robust command line tool for processing csv files?

I work with CSV files and sometimes need to quickly check the contents of a row or column from the command line. In many cases cut, head, tail, and friends will do the job; however, cut cannot easily ...
-1
votes
1answer
220 views

search for a character in one column and replace value in another column of matched records

I have a requirement to find value 2 in column 4 and wherever it matches replace 6th column with '-'sign in the beginning suppose if i have csv file - file1 like this abc,0,def,1234,ijk ...
8
votes
2answers
304 views

Reindexing a large CSV file

I went through the answers in this helpful thread, but my problem seems to be different enough that I can't think of good answer (at least with sed). I have a large CSV file (200+ GB) with rows that ...
-1
votes
3answers
266 views

How can I select columns based on a control file with invalid column numbers?

I have to generate the file dynamically from the source file based on the below control file. control_file.txt 1,3,5,-1,8,-1,4 The control file contains the position of columns which I require ...
1
vote
2answers
128 views

Count the maximum character length for all the data fields in a csv file and output to txt

Given a CSV with many data fields (>50), how can I count the maximum character length for each data field and then export all the counts to a txt file? BTW, I want to ignore the first line of the file ...
3
votes
3answers
445 views

Determine maximum column length for every column in csv-file

To determine the maximum length of each column in a comma-separated csv-file I hacked together a bash-script. When I ran it on a linux system it produced the correct output, but I need it to run on OS ...
2
votes
2answers
434 views

Delete rows in a csv file based on a specific column value

I have a csv file with the following headers - Interaction_ID, PDB_ID, First_Residue, First_Chain, Second_Residue, Third_Residue, Third_Chain & Pattern: ...
4
votes
2answers
1k views

How to display TSV (csv) in console, when empty cells are missed by: `column -t -s $'\t' `

I have file with columns spearated with tab. I have file when some rows have empty cells (on begining, in middle). In such cases column -t -s $'\t' simply fails: Input: $ echo -e ...
10
votes
4answers
4k views

command to layout tab separated list nicely

Sometimes, I'm getting as an input tab separated list, which is not quite aligned, for instance var1 var2 var3 var_with_long_name_which_ruins_alignment var2 var3 Is there an easy way to render ...
4
votes
5answers
9k views

extract the 4th column from a csv file using unix command

I have a csv file formatted as below. "col1","col2","col3","col4" "1","text1","<p>big html text</p> ","4th column" "2","text2","<p>big2 html2 text2</p> ...
-2
votes
2answers
321 views

Extract certain column from CSV files [closed]

I have a directory with a group of CSV files. All files have the same column headers. I need to extract values in a certain column from all files. The common part in files names is ...
0
votes
3answers
99 views

Remove all capital letters from a particular column in a CSV file

I am having a CSV file containing 3000+ lines in the following format: R1C1, ABC123abc, R1C3 R2C1, DEFG456defg, R2C3 R3C1, HIJKL789hijkl, R3C3 I need to remove all the capital letters from the ...
3
votes
1answer
576 views

How to export data in the terminal to a file (e.g. csv file)

I am a complete beginner in using Linux and I would like to know how to export a bunch of data in the command line to a csv or spreadsheet file (or even text would work). Right now I am simply using ...
1
vote
0answers
474 views

How to create CSV file without unnecessary spaces

I use the xls2csv binary in order to translate XLS doc to CSV in my linux red-hat machine, example: ( from man page ) xls2csv -x "1252spreadsheet.xls" -b WINDOWS-1252 -c "ut8csvfile.csv" -a UTF-8 ...
-3
votes
2answers
268 views

Shell Script to capture file name and file size in CSV format [closed]

I have number of zip files in a certain folder. I want to capture file name and the size of the file in a csv format containing file names and sizes, one file per day. The file names do contain the ...
2
votes
2answers
512 views

CSV file + set CSV parameters and values in bash script

I have the following CSV file: Alabama,Alaska,Arizona,Arkansas,California,Colorado,Connecticut,Delaware,Florida,Georgia,Hawaii,Idaho,Illinois,Indiana,Iowa 1000,"1 0 0 1",1002,1002,1003,1004,1005,"1 0 ...
1
vote
2answers
282 views

merge csv files, escape comma inside quotes

I have 3 csv files I want to join by first column (id column) Each file has the same 3 columns Row example : id | timestamp | Name 3792318, 2014-07-15 00:00:00, "A, B" When I join ...