Files containing data arrange in a table, often with commas (hence Comma Separated Values), to separate columns. Rows are separated by newlines (but not all newlines are row separators as fields can be quoted to contain the separator newlines. Use this tag for full-fledged CSV data not the simpler ...

learn more… | top users | synonyms

0
votes
1answer
18 views

Appending to a CSV file after obtaining datum from a database

I can [finally] extract data from the MySQL DB my_database_query="SELECT * FROM my_database.database" mysql -uUSER -p'PASSWORD' << EndOfFile $my_database_query EndOfFile How can I append this ...
2
votes
1answer
41 views

Using fixed strings in regular expressions?

I am writing a bash script which accepts a list of CSV files as arguments and outputs e-mail addresses only found in the first file. To accomplish this, for each record in the first CSV file I look ...
1
vote
1answer
36 views

Switch Columns in .csv files so that they are all the same

I have 20 excel files having 6x6 rows and columns, where the first row and column are string headers. Each of these files has 4 same columns and row headers except 1 different. I want to know how I ...
0
votes
0answers
13 views

Any application scans for wifi connected devices with the RSSI and saves as csv?

I'm working on a project at which people are to be identified in a closed area through their wifi connected devices. I accordingly need to find an application, code, or command that helps me scan for ...
1
vote
1answer
25 views

Convert Tab Separated csv using unoconv

I am attempting to convert a tab separated csv into an xls file using unoconv. However, the tabs are not being recognized as the field separator. Can someone please tell me how this can be done. I've ...
2
votes
1answer
26 views

Match and delete lines with ~100 strings

I have a list of roughly 100 entries to be deleted from a csv-delimited file. They are already in another text file called 'tbd.txt; My first thought is to write a bash for loop around 'sed -i' but ...
0
votes
2answers
46 views

I want to print the value from csv file dynamically

I want to print the values from properties file dynamically I am using below code but getting wrong output. testProperty.properties edmcws,edmbws,edmwt Code ...
2
votes
6answers
125 views

Scripting to split a single CSV row into multiple

I need a script that can take a CSV file with that has a column of semicolon-delimited attributes, and break it into multiple rows to normalize that multi-valued attribute? Here is the scenario: ...
0
votes
0answers
13 views

how to keep the spaces when inserting line to parameter [duplicate]

When I want to see for example the line number 2 , from the CSV file I get the following line example1 LINE_NUM=2 # sed -n "${LINE_NUM}p" file.csv cwdfwefwe,"aaaaa ","\dsw\dawd\aw ...
-3
votes
1answer
33 views

Empty columns which were seen before

I have a large CSV file and want to empty certain columns if they were seen before. So I have (to illustrate my problem): Category | Subcategory ---------+------------ foo | bar foo | bar ...
0
votes
1answer
76 views

csv file with multiple columns to zenity

How could I output the contents of a .csv file to a Zenity list. The file has multiple rows and columns and the rows are no fixed numbers. I would also like it so that when the user double clicks, on ...
-4
votes
1answer
191 views

Merge two csvfiles by column header

I have two csv files: success.csv "ID","Legacy ID","Field1", "Field2" 1,1111,google,news 2,2222,yahoo,news error.csv "Legacy ID","Field1","Field2","Message" 3333,aol,news,"Failed to upload ...
1
vote
2answers
37 views

change non-zero padded hour and minute into 24hr time

I have data that I need to change before running it through the pandas library in python. It is currently in a format that stores midnight values as 2400 and that should be changed to 0000. The ...
0
votes
0answers
16 views

Gzip Mutliple files using single command [duplicate]

Want to Gzip my all the .csv files into a gzip file. I am using this command. gzip *.csv; Problem File contains '//' in its name that's why command is giving error. Error : - '//File.csv' is not ...
2
votes
2answers
558 views

Sorting a csv file [duplicate]

I need to sort a CSV file, but the the header row (1st row) keeps getting sorted. This is what I'm using: cat data1.csv | sort -t"|" -k 1 -o data1.csv Here's a sample line: ...
0
votes
1answer
157 views

Is there a standalone tool which will write reports from CSV data files

I download CSV files (e.g. my bank account transactions) and would like to build reports (to print) from the data. I have searched and found several report writer libraries, but all would force me to ...
-1
votes
1answer
68 views

insert comma after the last letter

I use a txt file, and I need to convert it in csv file Saint Petersburg 0 10 0.1 - N Moscow - 9 0 - N Novgorod 0 7 1 30 Y In ...
0
votes
1answer
336 views

export from MYSQL to CSV

How do I export from MYSQL to CSV in mariadb on Centos 7? I am using: Server version 5.5.37-MariaDB Protocol version 10 Connection Localhost via UNIX socket UNIX socket ...
2
votes
1answer
305 views

splitting a CSV and keeping the header without intermediate files [closed]

I am trying to split a dozen 100MB+ csv files into managable smaller files for a curl post. I have managed to do it but with a lot of temporary files and IO. It's taking an eternity. I am hoping ...
1
vote
1answer
899 views

Confused about how to convert XML to CSV using xmlstarlet on OS X?

I asked a question earlier and it was closed as being a duplicate: How to parse XML to CSV with a shell script?. The answer in the duplicate that was linked suggested using a tool called XMLStarlet, ...
6
votes
4answers
2k views

Using jq to extract values and format in CSV

I have the below json file: { "data": [ { "displayName": "First Name", "rank": 1, "value": "VALUE" }, { "displayName": "Last Name", "rank": 2, ...
0
votes
1answer
985 views

Save file as Tab delimited text and spreadsheet

I am using the following command line in Linux to save the content of my input file (txt file consisted of columns) as a spreadsheet: less input_file > out_put.csv my output file is The ...
1
vote
1answer
355 views

How can I convert this excel file so that it is not only one row?

I've got an Excel file, shown in the picture below, and available for download here. What I need is to extract the variables under Item (Column B) and the values in column G. As a start, I tried ...
0
votes
1answer
95 views

Cut a specific part of a column from a csv file [closed]

I have to process a csv containing about 15 columns one of which contains a particular timestamp that includes milliseconds. When I go to insert this csv into a DB, I get an error because of this ...
9
votes
2answers
321 views

Reindexing a large CSV file

I went through the answers in this helpful thread, but my problem seems to be different enough that I can't think of good answer (at least with sed). I have a large CSV file (200+ GB) with rows that ...
-1
votes
3answers
342 views

How can I select columns based on a control file with invalid column numbers?

I have to generate the file dynamically from the source file based on the below control file. control_file.txt 1,3,5,-1,8,-1,4 The control file contains the position of columns which I require ...
2
votes
6answers
2k views

Handling comma in string values in a CSV file [duplicate]

I have a comma-separated file which has numeric and string columns. String columns are quoted and can have comma in between the quotes. How do I identify the columns with FS =","? Sample records ...
-2
votes
2answers
550 views

Extract certain column from CSV files [closed]

I have a directory with a group of CSV files. All files have the same column headers. I need to extract values in a certain column from all files. The common part in files names is ...
4
votes
4answers
477 views

Convert key=value blocks to CSV

I am trying to transpose a content of a file into another. I am new to shell scripting any help would be really appreciated Input file Test.txt: HLRSN = 3 IMSI = 404212109727229 KIVALUE = ...
0
votes
3answers
122 views

Remove all capital letters from a particular column in a CSV file

I am having a CSV file containing 3000+ lines in the following format: R1C1, ABC123abc, R1C3 R2C1, DEFG456defg, R2C3 R3C1, HIJKL789hijkl, R3C3 I need to remove all the capital letters from the ...
3
votes
1answer
1k views

How to export data in the terminal to a file (e.g. csv file)

I am a complete beginner in using Linux and I would like to know how to export a bunch of data in the command line to a csv or spreadsheet file (or even text would work). Right now I am simply using ...
1
vote
0answers
730 views

How to create CSV file without unnecessary spaces

I use the xls2csv binary in order to translate XLS doc to CSV in my linux red-hat machine, example: ( from man page ) xls2csv -x "1252spreadsheet.xls" -b WINDOWS-1252 -c "ut8csvfile.csv" -a UTF-8 ...
2
votes
2answers
796 views

CSV file + set CSV parameters and values in bash script

I have the following CSV file: Alabama,Alaska,Arizona,Arkansas,California,Colorado,Connecticut,Delaware,Florida,Georgia,Hawaii,Idaho,Illinois,Indiana,Iowa 1000,"1 0 0 1",1002,1002,1003,1004,1005,"1 0 ...
1
vote
2answers
339 views

merge csv files, escape comma inside quotes

I have 3 csv files I want to join by first column (id column) Each file has the same 3 columns Row example : id | timestamp | Name 3792318, 2014-07-15 00:00:00, "A, B" When I join ...
1
vote
2answers
304 views

Merge csv files with conditions

I have 2 CSV files for each date (csv_2014_4_15 and csv_2014_4_16) , with basic structure and couple of unique columns as below. id,name,created_at,updated_at,other columns 12, joe, 2013-1-1 18:30, ...
3
votes
1answer
1k views

merge csv files by first column

I have 3 csv files like this. csv 1: 1,aaaa,bbb,2014-04-01 2,qwe,rty,2014-04-03 3,zxc,cvb,2014-04-05 csv 2: 2,j,k,2014-04-01 3,a,s,2014-04-04 5,g,h,2014-04-08 csv 3: 2,a,s,d,f,g,2014-04-01 ...
3
votes
4answers
870 views

Concatenate filename inside of CSV file for each line for multiple CSV files

I have lots of CSV files and I need the filename of each specific CSV file in every line of each file. Original file content of abc123.csv: ColVal_1;ColVal_2;ColVal_3 ColVal_4;ColVal_5;ColVal_6 ...
0
votes
3answers
53 views

how to export all numbers that are unique in few text files into another file?

I have a.txt,b.txt,c.txt. Each has different numbers as below: a.txt: 12 14 111 1 15 2 b.txt 12 18 22 23 1 2 c.txt 12 14 15 16 17 1200 The output should contains all the numbers from ...
0
votes
2answers
52 views

Trim last chars from a line

I have a csv file and the lines have different length. But all of them ends with a number surrounded by a pair of braces. For example, ........{5} ........{16} I'd like to remove the number and the ...
1
vote
2answers
1k views

Shell script to Capture the file name and size in csv file and add header on that file

I have a number of zip files in a certain folder. I want to capture file name and the size of the files in a CSV format in day wise and put it in crontab so I can report on a daily basis, and then ...
-3
votes
2answers
308 views

Shell Script to capture file name and file size in CSV format [closed]

I have number of zip files in a certain folder. I want to capture file name and the size of the file in a csv format containing file names and sizes, one file per day. The file names do contain the ...
2
votes
3answers
1k views

replace column of a csv file with column from another file

I have two sample files like this: $ cat file1 abc,sachin cat,kumar $ cat file2 xyz pressure $ cat file3 xyz,sachin pressure, kumar I want the first column of file1 to be replaced with file2. ...
8
votes
3answers
2k views

Remove only the commas present within the double quotes

In a text file, I want to remove , (commas) and also the " (quotes) (only if the double quotes contains numbers seperated by commas). 56,72,"12,34,54",x,y,"foo,a,b,bar" Expected ouput ...
11
votes
8answers
1k views

How do I get the character count of words in a particular column?

I have a CSV file like this: abd,123,egypt,78 cde,456,england,45 How can I get the character count of only the 3rd column words? I can't figure out how to get wc to do this.
2
votes
3answers
5k views

Replacing the values in one file with the values in another file in bash

I am having a csv file naming List.csv in following format: Location,IP Address,Host Name,Domain,Domain Name, User Name,Manufacturer,Model,System Type, Serial Number, Operating System,RAM ...
3
votes
1answer
54 views

update line based on previous line

I have a issue since I'm not familiar with awk. I have csv file generated from sar -d output converted to csv style: 12:33:41 unix,restarts 12:35:00,lofi4096,0,0.0,0,0,0.0,0.0 ...
2
votes
2answers
2k views

Find the maximum length of characters in a field

Can you tell me how to find the maximum length of characters in a given field? For instance, with this input file: s.no,name 1,ATM 35,money 63,back The maximum length in field 1 is 2. So, I want ...
2
votes
1answer
427 views

solaris awk problems with date and if statements

I have daily files that come in via FTP with incorrect dates in the first column of the file. I have figured out how to deduct one day to derive the correct date and print this to a new file. The main ...
1
vote
5answers
84 views

Replicate a column in a one-line CSV file

I have a file that consists of a single line: a,x,b,c,d,e I want to convert this into a,x,b,x,c,x,d,x,e,x Which is the easiest way to achieve this? A solution based on Python will be most ...
1
vote
1answer
900 views

Removal of extra spaces but not the newline from csv files

I have to compare 2 files by using comm command like the below comm -23 csv1 csv2 > csv3 I seem to face the problem that both the csv are generated from different sources (csv1 is extracted ...