Files containing data arrange in a table, often with commas (hence Comma Separated Values), to separate columns. Rows are separated by newlines (but not all newlines are row separators as fields can be quoted to contain the separator newlines. Use this tag for full-fledged CSV data not the simpler ...

learn more… | top users | synonyms

3
votes
3answers
28 views

How can I change the first line of a big gzip file without decompressing all of it?

I currently have a compressed file, A.gz which contains lots of tabulated data, including a header in the first line. I want to create another file, B.gz, which has the same data as the previous file, ...
1
vote
2answers
44 views

How do fast Convert a .xls (MS Excel) file to .csv

How will do fast covert excel to csv. I had tried with php, perl, JavaScript. but it will take too much time. i.e for 5MB file it will take 3-5 mints
3
votes
2answers
1k views

Print some fields of /etc/passwd

I have a question concerning the usage of wild cards in Linux. Here I have some lines from a long list of lines in the /etc/passwd -file. root:x:0:0:root:/root:/bin/bash ...
3
votes
2answers
441 views

Need shell script to transform a CSV into Apache httpd format

Need some pointing in right direction on script to fetch and regex or sed. Site24x7 provides a URL with a CSV list of their source IP's used for monitoring. (they also provide other formats, CSV ...
0
votes
0answers
28 views

Convert vnstat log file to csv

My goal is to log daily network activity (rx/tx) from each ethernet interface. I installed vnstat, and while it reports daily and monthly stats, it doesn't offer a nice way to convert to csv to ...
2
votes
2answers
6k views

Convert csv to HTML table

I have a Medical.csv file with rows of following format, field: 'participation.type', displayName: 'program_type', type: 'String',path:'participation' field: 'participation.program', ...
10
votes
6answers
457 views

Remove lines from a file depending on lines found in another file

File file1.txt contains lines like: /api/purchase/<hash>/index.html For example: /api/purchase/12ab09f46/index.html File file2.csv contains lines like: <hash>,timestamp,ip_address ...
1
vote
3answers
56 views

Process text file - Extract data starting with number

I have a list of contact information I need to process in .txt format seperated by , . We only want to keep SIC codes that start with 65 which is real estate. The command should only check for the ...
17
votes
4answers
52k views

Convert a .xlsx (MS Excel) file to .csv on command line with semicolon separated fields

I realize that this is not an entirely unix/linux related question. But since this is something I'll do on linux, I hope someone has an answer. I have an online excel file (.xlsx) which gets updated ...
0
votes
2answers
82 views

Open CSV File And Go Straight To Spreadsheet

Part of my software issues various commands to open and view different file types. For instance I use atril for PDFs and eom for PNGs. However I have a slight problem with CSV files. I can open them ...
3
votes
5answers
125 views

Efficient data extraction from multiple files to a single CSV file

I have a large collection of XML files with the same exact structure: $ cat file_<ID>.xml ... ... ... <double>1.2342</double> <double>2.3456</double> ...
0
votes
1answer
17 views

How to install and run xlsx2csv on Fedora 19?

I was not able to run xlsx2csv at the command line on Fedora 19 [ec2-user@localhost Downloads]$ xlsx2csv bash: xlsx2csv: command not found... So I looked to install the corresponding RPM package ...
2
votes
3answers
116 views

How to sum column values for each row in two csv files using bash script?

I have two CSV files and i have the requirement to calculate the sum of the 8th column from File1 and File2 for every row. The two CSV files have exactly 24 rows each and to simplify the things i have ...
6
votes
3answers
8k views

Filter a .CSV file based on the 5th column values of a file and print those records into a new file

I have a .CSV file with the below format: "column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10 "12310","42324564756","a simple string with a , ...
3
votes
3answers
159 views

If statement string comparison

I'm reading a csv file with 14 values per line. I want to convert the HEX value from 10th [f10] value to a DEC value. In order to do this if the 7th [f7] value must hold EnHr or EnSt or SpJb or Chem ...
0
votes
4answers
181 views

How to extract column name (header) from a CSV file which contains the max value in a row?

I am trying to extract the column name with the maximum value in each row using bash script i.e., the column header value or the value from the same column in the first row. I am using the following ...
0
votes
2answers
46 views

How can I combine different csv files from differenet folders with the name of each file as a column name in the combined file?

I have a bunch of directories in which there are 3 .csv files with different names. For example, in my directories aa bb cc dd there are 3 files in each: aa: EA_sing_aa.csv EA_ska_aa.csv ...
1
vote
2answers
126 views

Remove lines from CSV file if the second column is in a word list

I have found a long list of free email providers that I want to remove from my email lists - https://gist.github.com/tbrianjones/5992856 Below are two commands I currently use that do the same job ...
5
votes
3answers
87 views

update line based on previous line

I have a issue since I'm not familiar with awk. I have csv file generated from sar -d output converted to csv style: 12:33:41 unix,restarts 12:35:00,lofi4096,0,0.0,0,0,0.0,0.0 ...
12
votes
5answers
665 views

How can I count the number of lines in a file after a grep match?

I am trying to count the number of lines after a problematic row in a csv file. I am aware I can use the grep -a # syntax to output # number of lines after a match has been found. I'm only interested ...
3
votes
4answers
6k views

Extracting column from comma separated text

I have a long comma-separated delimited file with 20K lines. Here's a sample: "","id","number1","number2","number3","number4","number5","number6","number7" ...
2
votes
4answers
58 views

CSV - Only keep certain entries

I have a bunch of data inside "" and separated by , - Example below of two lines "stampthisandthat.com","GANDI ...
2
votes
5answers
294 views

Duplicate lines and make them unique?

I have the following structure of text lines: 3923 001 L05 LV,L05 RM 3923 002 L12 RA,L12 LA 3923 003 I06 ALL 3923 004 G04 RV,Z09 ALL but i would need this: 3923 001 L05 LV 3923 001 ...
1
vote
1answer
114 views

How can i use multiple if statement inside another if statement of a awk program?

I have a CSV File and i want to have multiple checks to retrieve the data from the CSV file. I have the following conditions which needs to be taken care of and i am trying to use awk in a shell ...
0
votes
2answers
54 views

Merging large data set in fastest possible way

I have a large data set ~100 GBs having smaller csv files ~ 100,000. I wish to merge all .csv files together in the fastest possible way. Files are located in two different folders. Someone ...
0
votes
0answers
90 views

extract text to csv file from output of reading multiple files into specific rows

I am trying to write a bash script to generate a csv file from text within multiple pdf documents. I have a script for converting pdf to text, but not for generating the csv file. Each text document ...
1
vote
1answer
141 views

How to select rows from a CSV file based on different column values?

I have a CSV file with values like the following: col1,col2,col3,col4,col5,col6,col7 1,0,0,0,0,BTS,Active 4,5,3,1,1,LocalMode,Offlne 1,2,5,6,3,PermFault,Offline 1,2,6,6,2,BSC,Active ...
2
votes
2answers
413 views

Merge csv files with conditions

I have 2 CSV files for each date (csv_2014_4_15 and csv_2014_4_16) , with basic structure and couple of unique columns as below. id,name,created_at,updated_at,other columns 12, joe, 2013-1-1 18:30, ...
1
vote
1answer
48 views

How to parse a CSV file based with groups made of specific number of rows?

I have a CSV file which has the following format: Date,Towns,Count1,Count2 6/26/2015,HYDERABAD,234552,0 ,KARIM NAGAR,7520,0 ,RAJAMUNDRY,7248,256 ,KURNOOL,5848,32,0 ...
1
vote
1answer
87 views

Pick columns from a variable length csv file

I have two CSV files, which share a column that is unique to each row in each file, like an ID. The files don't have a header. The file_2 has variable length columns e.g. 1,2,3 4,5,6,7,8 I joined ...
24
votes
11answers
12k views

Is there a robust command line tool for processing csv files?

I work with CSV files and sometimes need to quickly check the contents of a row or column from the command line. In many cases cut, head, tail, and friends will do the job; however, cut cannot easily ...
7
votes
4answers
5k views

Using jq to extract values and format in CSV

I have the below JSON file: { "data": [ { "displayName": "First Name", "rank": 1, "value": "VALUE" }, { "displayName": "Last Name", "rank": 2, ...
1
vote
2answers
53 views

Alignment when printing a series of records

How can I resolve the problem of the tables going to the right. I just want it to be shown under 1. Here's my script with START added. The alignment has gone wonky now: while IFS="," read f1 f2 f3 ...
5
votes
3answers
165 views

Compare two files with four columns

I have two csv files with some log information. I need to compare line to line the field1 (file1) with field2 (file2) with a range of 1 minute (-00:00:01 < time < 00:00:01) and the field5 ...
1
vote
1answer
2k views

Confused about how to convert XML to CSV using xmlstarlet on OS X?

I asked a question earlier and it was closed as being a duplicate: How to parse XML to CSV with a shell script?. The answer in the duplicate that was linked suggested using a tool called XMLStarlet, ...
1
vote
0answers
50 views

How to create a Kate highlight syntax to CSV files

I'm working with CSV files and I'm using vim because of the rainbow csv plugin that makes the editing easier: http://www.vim.org/scripts/script.php?script_id=4598 But I prefer using Kate and as I ...
0
votes
2answers
341 views

convert xml to csv

i need to convert a xml to csv using script. I found a solution with xmlstarlet but it is not available so i am returning to point 0. Here is my xml. <root> <record> ...
25
votes
4answers
7k views

sort inplace - like sed --in-place - exists?

Am I blind or is there no option like --in-place for sort? In order to save results to the input file, sed uses -i (--in-place). Redirecting the output of sort to the input file sort < f > f ...
0
votes
1answer
65 views

Appending to a CSV file after obtaining datum from a database

I can [finally] extract data from the MySQL DB my_database_query="SELECT * FROM my_database.database" mysql -uUSER -p'PASSWORD' << EndOfFile $my_database_query EndOfFile How can I append this ...
2
votes
1answer
62 views

Using fixed strings in regular expressions?

I am writing a bash script which accepts a list of CSV files as arguments and outputs e-mail addresses only found in the first file. To accomplish this, for each record in the first CSV file I look ...
1
vote
1answer
105 views

Switch Columns in .csv files so that they are all the same

I have 20 excel files having 6x6 rows and columns, where the first row and column are string headers. Each of these files has 4 same columns and row headers except 1 different. I want to know how I ...
0
votes
0answers
32 views

Any application scans for wifi connected devices with the RSSI and saves as csv?

I'm working on a project at which people are to be identified in a closed area through their wifi connected devices. I accordingly need to find an application, code, or command that helps me scan for ...
0
votes
1answer
477 views

Is there a standalone tool which will write reports from CSV data files

I download CSV files (e.g. my bank account transactions) and would like to build reports (to print) from the data. I have searched and found several report writer libraries, but all would force me to ...
2
votes
1answer
129 views

Convert Tab Separated csv using unoconv

I am attempting to convert a tab separated csv into an xls file using unoconv. However, the tabs are not being recognized as the field separator. Can someone please tell me how this can be done. I've ...
1
vote
0answers
1k views

How to create CSV file without unnecessary spaces

I use the xls2csv binary in order to translate XLS doc to CSV in my linux red-hat machine, example: ( from man page ) xls2csv -x "1252spreadsheet.xls" -b WINDOWS-1252 -c "ut8csvfile.csv" -a UTF-8 ...
2
votes
1answer
33 views

Match and delete lines with ~100 strings

I have a list of roughly 100 entries to be deleted from a csv-delimited file. They are already in another text file called 'tbd.txt; My first thought is to write a bash for loop around 'sed -i' but ...
0
votes
2answers
326 views

I want to print the value from csv file dynamically

I want to print the values from properties file dynamically I am using below code but getting wrong output. testProperty.properties edmcws,edmbws,edmwt Code ...
3
votes
2answers
3k views

Swap two columns in a CSV using SED

I have a CSV file that contains 10 different fields (, is the deliminator). Example data: student-id,last,first,hwk1,hwk2,hwk3,exam1,hwk4,hwk5,exam2 pts-avail,,,100,150,100,200,150,100,300 ...
2
votes
6answers
244 views

Scripting to split a single CSV row into multiple

I need a script that can take a CSV file with that has a column of semicolon-delimited attributes, and break it into multiple rows to normalize that multi-valued attribute? Here is the scenario: ...
0
votes
0answers
13 views

how to keep the spaces when inserting line to parameter [duplicate]

When I want to see for example the line number 2 , from the CSV file I get the following line example1 LINE_NUM=2 # sed -n "${LINE_NUM}p" file.csv cwdfwefwe,"aaaaa ","\dsw\dawd\aw ...