Files containing data arrange in a table, often with commas (hence Comma Separated Values), to separate columns. Rows are separated by newlines (but not all newlines are row separators as fields can be quoted to contain the separator newlines. Use this tag for full-fledged CSV data not the simpler ...

learn more… | top users | synonyms

0
votes
0answers
28 views

Convert vnstat log file to csv

My goal is to log daily network activity (rx/tx) from each ethernet interface. I installed vnstat, and while it reports daily and monthly stats, it doesn't offer a nice way to convert to csv to ...
1
vote
3answers
56 views

Process text file - Extract data starting with number

I have a list of contact information I need to process in .txt format seperated by , . We only want to keep SIC codes that start with 65 which is real estate. The command should only check for the ...
10
votes
6answers
456 views

Remove lines from a file depending on lines found in another file

File file1.txt contains lines like: /api/purchase/<hash>/index.html For example: /api/purchase/12ab09f46/index.html File file2.csv contains lines like: <hash>,timestamp,ip_address ...
1
vote
2answers
42 views

How do fast Convert a .xls (MS Excel) file to .csv

How will do fast covert excel to csv. I had tried with php, perl, JavaScript. but it will take too much time. i.e for 5MB file it will take 3-5 mints
0
votes
2answers
81 views

Open CSV File And Go Straight To Spreadsheet

Part of my software issues various commands to open and view different file types. For instance I use atril for PDFs and eom for PNGs. However I have a slight problem with CSV files. I can open them ...
2
votes
3answers
115 views

How to sum column values for each row in two csv files using bash script?

I have two CSV files and i have the requirement to calculate the sum of the 8th column from File1 and File2 for every row. The two CSV files have exactly 24 rows each and to simplify the things i have ...
0
votes
1answer
17 views

How to install and run xlsx2csv on Fedora 19?

I was not able to run xlsx2csv at the command line on Fedora 19 [ec2-user@localhost Downloads]$ xlsx2csv bash: xlsx2csv: command not found... So I looked to install the corresponding RPM package ...
3
votes
3answers
158 views

If statement string comparison

I'm reading a csv file with 14 values per line. I want to convert the HEX value from 10th [f10] value to a DEC value. In order to do this if the 7th [f7] value must hold EnHr or EnSt or SpJb or Chem ...
0
votes
2answers
46 views

How can I combine different csv files from differenet folders with the name of each file as a column name in the combined file?

I have a bunch of directories in which there are 3 .csv files with different names. For example, in my directories aa bb cc dd there are 3 files in each: aa: EA_sing_aa.csv EA_ska_aa.csv ...
0
votes
4answers
180 views

How to extract column name (header) from a CSV file which contains the max value in a row?

I am trying to extract the column name with the maximum value in each row using bash script i.e., the column header value or the value from the same column in the first row. I am using the following ...
3
votes
5answers
124 views

Efficient data extraction from multiple files to a single CSV file

I have a large collection of XML files with the same exact structure: $ cat file_<ID>.xml ... ... ... <double>1.2342</double> <double>2.3456</double> ...
1
vote
2answers
125 views

Remove lines from CSV file if the second column is in a word list

I have found a long list of free email providers that I want to remove from my email lists - https://gist.github.com/tbrianjones/5992856 Below are two commands I currently use that do the same job ...
2
votes
4answers
58 views

CSV - Only keep certain entries

I have a bunch of data inside "" and separated by , - Example below of two lines "stampthisandthat.com","GANDI ...
2
votes
5answers
294 views

Duplicate lines and make them unique?

I have the following structure of text lines: 3923 001 L05 LV,L05 RM 3923 002 L12 RA,L12 LA 3923 003 I06 ALL 3923 004 G04 RV,Z09 ALL but i would need this: 3923 001 L05 LV 3923 001 ...
1
vote
1answer
113 views

How can i use multiple if statement inside another if statement of a awk program?

I have a CSV File and i want to have multiple checks to retrieve the data from the CSV file. I have the following conditions which needs to be taken care of and i am trying to use awk in a shell ...
0
votes
2answers
54 views

Merging large data set in fastest possible way

I have a large data set ~100 GBs having smaller csv files ~ 100,000. I wish to merge all .csv files together in the fastest possible way. Files are located in two different folders. Someone ...
1
vote
1answer
140 views

How to select rows from a CSV file based on different column values?

I have a CSV file with values like the following: col1,col2,col3,col4,col5,col6,col7 1,0,0,0,0,BTS,Active 4,5,3,1,1,LocalMode,Offlne 1,2,5,6,3,PermFault,Offline 1,2,6,6,2,BSC,Active ...
0
votes
0answers
90 views

extract text to csv file from output of reading multiple files into specific rows

I am trying to write a bash script to generate a csv file from text within multiple pdf documents. I have a script for converting pdf to text, but not for generating the csv file. Each text document ...
1
vote
1answer
48 views

How to parse a CSV file based with groups made of specific number of rows?

I have a CSV file which has the following format: Date,Towns,Count1,Count2 6/26/2015,HYDERABAD,234552,0 ,KARIM NAGAR,7520,0 ,RAJAMUNDRY,7248,256 ,KURNOOL,5848,32,0 ...
1
vote
1answer
87 views

Pick columns from a variable length csv file

I have two CSV files, which share a column that is unique to each row in each file, like an ID. The files don't have a header. The file_2 has variable length columns e.g. 1,2,3 4,5,6,7,8 I joined ...
12
votes
5answers
665 views

How can I count the number of lines in a file after a grep match?

I am trying to count the number of lines after a problematic row in a csv file. I am aware I can use the grep -a # syntax to output # number of lines after a match has been found. I'm only interested ...
1
vote
2answers
53 views

Alignment when printing a series of records

How can I resolve the problem of the tables going to the right. I just want it to be shown under 1. Here's my script with START added. The alignment has gone wonky now: while IFS="," read f1 f2 f3 ...
5
votes
3answers
165 views

Compare two files with four columns

I have two csv files with some log information. I need to compare line to line the field1 (file1) with field2 (file2) with a range of 1 minute (-00:00:01 < time < 00:00:01) and the field5 ...
1
vote
0answers
50 views

How to create a Kate highlight syntax to CSV files

I'm working with CSV files and I'm using vim because of the rainbow csv plugin that makes the editing easier: http://www.vim.org/scripts/script.php?script_id=4598 But I prefer using Kate and as I ...
0
votes
2answers
333 views

convert xml to csv

i need to convert a xml to csv using script. I found a solution with xmlstarlet but it is not available so i am returning to point 0. Here is my xml. <root> <record> ...
0
votes
1answer
65 views

Appending to a CSV file after obtaining datum from a database

I can [finally] extract data from the MySQL DB my_database_query="SELECT * FROM my_database.database" mysql -uUSER -p'PASSWORD' << EndOfFile $my_database_query EndOfFile How can I append this ...
2
votes
1answer
62 views

Using fixed strings in regular expressions?

I am writing a bash script which accepts a list of CSV files as arguments and outputs e-mail addresses only found in the first file. To accomplish this, for each record in the first CSV file I look ...
1
vote
1answer
105 views

Switch Columns in .csv files so that they are all the same

I have 20 excel files having 6x6 rows and columns, where the first row and column are string headers. Each of these files has 4 same columns and row headers except 1 different. I want to know how I ...
0
votes
0answers
32 views

Any application scans for wifi connected devices with the RSSI and saves as csv?

I'm working on a project at which people are to be identified in a closed area through their wifi connected devices. I accordingly need to find an application, code, or command that helps me scan for ...
2
votes
1answer
129 views

Convert Tab Separated csv using unoconv

I am attempting to convert a tab separated csv into an xls file using unoconv. However, the tabs are not being recognized as the field separator. Can someone please tell me how this can be done. I've ...
2
votes
1answer
33 views

Match and delete lines with ~100 strings

I have a list of roughly 100 entries to be deleted from a csv-delimited file. They are already in another text file called 'tbd.txt; My first thought is to write a bash for loop around 'sed -i' but ...
0
votes
2answers
324 views

I want to print the value from csv file dynamically

I want to print the values from properties file dynamically I am using below code but getting wrong output. testProperty.properties edmcws,edmbws,edmwt Code ...
2
votes
6answers
243 views

Scripting to split a single CSV row into multiple

I need a script that can take a CSV file with that has a column of semicolon-delimited attributes, and break it into multiple rows to normalize that multi-valued attribute? Here is the scenario: ...
0
votes
0answers
13 views

how to keep the spaces when inserting line to parameter [duplicate]

When I want to see for example the line number 2 , from the CSV file I get the following line example1 LINE_NUM=2 # sed -n "${LINE_NUM}p" file.csv cwdfwefwe,"aaaaa ","\dsw\dawd\aw ...
-3
votes
1answer
35 views

Empty columns which were seen before

I have a large CSV file and want to empty certain columns if they were seen before. So I have (to illustrate my problem): Category | Subcategory ---------+------------ foo | bar foo | bar ...
0
votes
1answer
179 views

csv file with multiple columns to zenity

How could I output the contents of a .csv file to a Zenity list. The file has multiple rows and columns and the rows are no fixed numbers. I would also like it so that when the user double clicks, on ...
-4
votes
1answer
302 views

Merge two csvfiles by column header

I have two csv files: success.csv "ID","Legacy ID","Field1", "Field2" 1,1111,google,news 2,2222,yahoo,news error.csv "Legacy ID","Field1","Field2","Message" 3333,aol,news,"Failed to upload ...
1
vote
2answers
62 views

change non-zero padded hour and minute into 24hr time

I have data that I need to change before running it through the pandas library in python. It is currently in a format that stores midnight values as 2400 and that should be changed to 0000. The ...
0
votes
0answers
16 views

Gzip Mutliple files using single command [duplicate]

Want to Gzip my all the .csv files into a gzip file. I am using this command. gzip *.csv; Problem File contains '//' in its name that's why command is giving error. Error : - '//File.csv' is not ...
1
vote
2answers
1k views

Sorting a csv file [duplicate]

I need to sort a CSV file, but the the header row (1st row) keeps getting sorted. This is what I'm using: cat data1.csv | sort -t"|" -k 1 -o data1.csv Here's a sample line: ...
0
votes
1answer
474 views

Is there a standalone tool which will write reports from CSV data files

I download CSV files (e.g. my bank account transactions) and would like to build reports (to print) from the data. I have searched and found several report writer libraries, but all would force me to ...
-1
votes
1answer
94 views

insert comma after the last letter

I use a txt file, and I need to convert it in csv file Saint Petersburg 0 10 0.1 - N Moscow - 9 0 - N Novgorod 0 7 1 30 Y In ...
1
vote
1answer
1k views

export from MYSQL to CSV

How do I export from MYSQL to CSV in mariadb on Centos 7? I am using: Server version 5.5.37-MariaDB Protocol version 10 Connection Localhost via UNIX socket UNIX socket ...
2
votes
1answer
691 views

splitting a CSV and keeping the header without intermediate files [closed]

I am trying to split a dozen 100MB+ csv files into managable smaller files for a curl post. I have managed to do it but with a lot of temporary files and IO. It's taking an eternity. I am hoping ...
1
vote
1answer
2k views

Confused about how to convert XML to CSV using xmlstarlet on OS X?

I asked a question earlier and it was closed as being a duplicate: How to parse XML to CSV with a shell script?. The answer in the duplicate that was linked suggested using a tool called XMLStarlet, ...
7
votes
4answers
5k views

Using jq to extract values and format in CSV

I have the below JSON file: { "data": [ { "displayName": "First Name", "rank": 1, "value": "VALUE" }, { "displayName": "Last Name", "rank": 2, ...
0
votes
1answer
2k views

Save file as Tab delimited text and spreadsheet

I am using the following command line in Linux to save the content of my input file (txt file consisted of columns) as a spreadsheet: less input_file > out_put.csv my output file is The ...
1
vote
1answer
899 views

How can I convert this excel file so that it is not only one row?

I've got an Excel file, shown in the picture below, and available for download here. What I need is to extract the variables under Item (Column B) and the values in column G. As a start, I tried ...
0
votes
1answer
129 views

Cut a specific part of a column from a csv file [closed]

I have to process a csv containing about 15 columns one of which contains a particular timestamp that includes milliseconds. When I go to insert this csv into a DB, I get an error because of this ...
10
votes
2answers
356 views

Reindexing a large CSV file

I went through the answers in this helpful thread, but my problem seems to be different enough that I can't think of good answer (at least with sed). I have a large CSV file (200+ GB) with rows that ...