Comma-separated values: files containing data arrange in a table, with commas to separate columns

learn more… | top users | synonyms

1
vote
1answer
13 views

Confused about how to convert XML to CSV using xmlstarlet on OS X?

I asked a question earlier and it was closed as being a duplicate: How to parse XML to CSV with a shell script?. The answer in the duplicate that was linked suggested using a tool called XMLStarlet, ...
0
votes
0answers
16 views

Convert XML to CSV using the command line [duplicate]

How can i do this? Heres an example: (no real data) <text> <Caption>Hello</Caption> <Comment>Hello-Again</Comment> ...
4
votes
4answers
86 views

Using jq to extract values and format in CSV

I have the below json file: { "data": [ { "displayName": "First Name", "rank": 1, "value": "VALUE" }, { "displayName": "Last Name", "rank": 2, ...
0
votes
1answer
37 views

Save file as Tab delimited text and spreadsheet

I am using the following command line in Linux to save the content of my input file (txt file consisted of columns) as a spreadsheet: less input_file > out_put.csv my output file is The ...
0
votes
1answer
57 views

Cut a specific part of a column from a csv file [closed]

I have to process a csv containing about 15 columns one of which contains a particular timestamp that includes milliseconds. When I go to insert this csv into a DB, I get an error because of this ...
-1
votes
1answer
35 views

search for a character in one column and replace value in another column of matched records

I have a requirement to find value 2 in column 4 and wherever it matches replace 6th column with '-'sign in the beginning suppose if i have csv file - file1 like this abc,0,def,1234,ijk ...
8
votes
2answers
276 views

Reindexing a large CSV file

I went through the answers in this helpful thread, but my problem seems to be different enough that I can't think of good answer (at least with sed). I have a large CSV file (200+ GB) with rows that ...
-1
votes
3answers
114 views

How can I select columns based on a control file with invalid column numbers?

I have to generate the file dynamically from the source file based on the below control file. control_file.txt 1,3,5,-1,8,-1,4 The control file contains the position of columns which I require ...
1
vote
2answers
61 views

Count the maximum character length for all the data fields in a csv file and output to txt

Given a CSV with many data fields (>50), how can I count the maximum character length for each data field and then export all the counts to a txt file? BTW, I want to ignore the first line of the file ...
3
votes
3answers
167 views

Determine maximum column length for every column in csv-file

To determine the maximum length of each column in a comma-separated csv-file I hacked together a bash-script. When I ran it on a linux system it produced the correct output, but I need it to run on OS ...
2
votes
2answers
81 views

Delete rows in a csv file based on a specific column value

I have a csv file with the following headers - Interaction_ID, PDB_ID, First_Residue, First_Chain, Second_Residue, Third_Residue, Third_Chain & Pattern: ...
2
votes
6answers
562 views

Handling comma in string values in a CSV file [duplicate]

I have a comma-separated file which has numeric and string columns. String columns are quoted and can have comma in between the quotes. How do I identify the columns with FS =","? Sample records ...
-3
votes
2answers
133 views

Extract certain column from CSV files [closed]

I have a directory with a group of CSV files. All files have the same column headers. I need to extract values in a certain column from all files. The common part in files names is ...
4
votes
4answers
289 views

Convert key=value blocks to CSV

I am trying to transpose a content of a file into another. I am new to shell scripting any help would be really appreciated Input file Test.txt: HLRSN = 3 IMSI = 404212109727229 KIVALUE = ...
0
votes
3answers
70 views

Remove all capital letters from a particular column in a CSV file

I am having a CSV file containing 3000+ lines in the following format: R1C1, ABC123abc, R1C3 R2C1, DEFG456defg, R2C3 R3C1, HIJKL789hijkl, R3C3 I need to remove all the capital letters from the ...
3
votes
1answer
164 views

How to export data in the terminal to a file (e.g. csv file)

I am a complete beginner in using Linux and I would like to know how to export a bunch of data in the command line to a csv or spreadsheet file (or even text would work). Right now I am simply using ...
1
vote
0answers
181 views

How to create CSV file without unnecessary spaces

I use the xls2csv binary in order to translate XLS doc to CSV in my linux red-hat machine, example: ( from man page ) xls2csv -x "1252spreadsheet.xls" -b WINDOWS-1252 -c "ut8csvfile.csv" -a UTF-8 ...
2
votes
2answers
206 views

CSV file + set CSV parameters and values in bash script

I have the following CSV file: Alabama,Alaska,Arizona,Arkansas,California,Colorado,Connecticut,Delaware,Florida,Georgia,Hawaii,Idaho,Illinois,Indiana,Iowa 1000,"1 0 0 1",1002,1002,1003,1004,1005,"1 0 ...
1
vote
2answers
186 views

merge csv files, escape comma inside quotes

I have 3 csv files I want to join by first column (id column) Each file has the same 3 columns Row example : id | timestamp | Name 3792318, 2014-07-15 00:00:00, "A, B" When I join ...
1
vote
2answers
155 views

Merge csv files with conditions

I have 2 CSV files for each date (csv_2014_4_15 and csv_2014_4_16) , with basic structure and couple of unique columns as below. id,name,created_at,updated_at,other columns 12, joe, 2013-1-1 18:30, ...
3
votes
1answer
302 views

merge csv files by first column

I have 3 csv files like this. csv 1: 1,aaaa,bbb,2014-04-01 2,qwe,rty,2014-04-03 3,zxc,cvb,2014-04-05 csv 2: 2,j,k,2014-04-01 3,a,s,2014-04-04 5,g,h,2014-04-08 csv 3: 2,a,s,d,f,g,2014-04-01 ...
2
votes
3answers
260 views

Concatenate filename inside of CSV file for each line for multiple CSV files

I have lots of CSV files and I need the filename of each specific CSV file in every line of each file. Original file content of abc123.csv: ColVal_1;ColVal_2;ColVal_3 ColVal_4;ColVal_5;ColVal_6 ...
0
votes
3answers
51 views

how to export all numbers that are unique in few text files into another file?

I have a.txt,b.txt,c.txt. Each has different numbers as below: a.txt: 12 14 111 1 15 2 b.txt 12 18 22 23 1 2 c.txt 12 14 15 16 17 1200 The output should contains all the numbers from ...
0
votes
2answers
35 views

Trim last chars from a line

I have a csv file and the lines have different length. But all of them ends with a number surrounded by a pair of braces. For example, ........{5} ........{16} I'd like to remove the number and the ...
1
vote
2answers
378 views

Shell script to Capture the file name and size in csv file and add header on that file

I have a number of zip files in a certain folder. I want to capture file name and the size of the files in a CSV format in day wise and put it in crontab so I can report on a daily basis, and then ...
-3
votes
2answers
219 views

Shell Script to capture file name and file size in CSV format [closed]

I have number of zip files in a certain folder. I want to capture file name and the size of the file in a csv format containing file names and sizes, one file per day. The file names do contain the ...
2
votes
3answers
660 views

replace column of a csv file with column from another file

I have two sample files like this: $ cat file1 abc,sachin cat,kumar $ cat file2 xyz pressure $ cat file3 xyz,sachin pressure, kumar I want the first column of file1 to be replaced with file2. ...
6
votes
3answers
772 views

Remove only the commas present within the double quotes

In a text file, I want to remove , (commas) and also the " (quotes) (only if the double quotes contains numbers seperated by commas). 56,72,"12,34,54",x,y,"foo,a,b,bar" Expected ouput ...
11
votes
8answers
1k views

How do I get the character count of words in a particular column?

I have a CSV file like this: abd,123,egypt,78 cde,456,england,45 How can I get the character count of only the 3rd column words? I can't figure out how to get wc to do this.
2
votes
3answers
3k views

Replacing the values in one file with the values in another file in bash

I am having a csv file naming List.csv in following format: Location,IP Address,Host Name,Domain,Domain Name, User Name,Manufacturer,Model,System Type, Serial Number, Operating System,RAM ...
3
votes
1answer
46 views

update line based on previous line

I have a issue since I'm not familiar with awk. I have csv file generated from sar -d output converted to csv style: 12:33:41 unix,restarts 12:35:00,lofi4096,0,0.0,0,0,0.0,0.0 ...
1
vote
2answers
860 views

Find the maximum length of characters in a field

Can you tell me how to find the maximum length of characters in a given field? For instance, with this input file: s.no,name 1,ATM 35,money 63,back The maximum length in field 1 is 2. So, I want ...
2
votes
1answer
286 views

solaris awk problems with date and if statements

I have daily files that come in via FTP with incorrect dates in the first column of the file. I have figured out how to deduct one day to derive the correct date and print this to a new file. The main ...
1
vote
5answers
77 views

Replicate a column in a one-line CSV file

I have a file that consists of a single line: a,x,b,c,d,e I want to convert this into a,x,b,x,c,x,d,x,e,x Which is the easiest way to achieve this? A solution based on Python will be most ...
1
vote
1answer
674 views

Removal of extra spaces but not the newline from csv files

I have to compare 2 files by using comm command like the below comm -23 csv1 csv2 > csv3 I seem to face the problem that both the csv are generated from different sources (csv1 is extracted ...
6
votes
3answers
515 views

adding an empty first line inside csv file

I input data into a csv by 'echo'-ing some values into another csv. I tried echo "" and in the next line echo "values" but i get intermittent empty lines.. I need empty line only at the start....
2
votes
2answers
189 views

Create an ASCII art table from tabular data

Given perhaps comma- or tab-delimited input, I'd like to present a series of appropriately padded columns to stdout, so I can easily scan columnar information which would otherwise present rather ...
0
votes
4answers
795 views

Removing certain fields from a CSV file

I have a large CSV file (~15GB) which a client has sent over as an export of a table. The format is a follows: field, ... , field, field which sometimes contains commas, another field which sometimes ...
2
votes
3answers
324 views

Extract data from a file and place in different files based on1 column value

We will generate a csv file with below values yp1234,577,1,3 yp5678,577,3,5 yp9012,132,8,9 I need to extract data and create files based on second column. If it's 577 then the whole line has to be ...
4
votes
2answers
75 views

awk pairwise differences

I run into this problem a lot with data, if I have some integers, I frequently want to look at differences between adjacent integers, I usually solve this in Ruby or Python, but I think such a thing ...
2
votes
3answers
285 views

I need data from a csv file which is not present in the second csv file

For example: I have a CSV file (first file ) with below data (vertically): 1 2 3 4 5 6 Second CSV with below data (vertically): 5 6 7 8 I want my resultant file (action needs to be performed ...
2
votes
3answers
1k views

Select lines from text file which have ids listed in another file

I use a lot of grep awk sort in my unix shell to work with medium-sized (around 10M-100M lines) tab-separated column text files. In this respect unix shell is my spreadsheet. But I have one huge ...
1
vote
1answer
134 views

removing data from txt file

I have a comma-delimited giant file that was exported from a database. The file has this format Date,n1,n2,n3,n4,n5,n6,color,type,reference 10-Jan-2000,344,13,21,2,55,3,"rose garden" ,3444,32 ...
1
vote
2answers
189 views

Unable to do head or tail for a file

I have a .CSV file which upon passing the file test_file.csv command gives the output as: test_file.csv: ISO-8859 English text, with CR line terminators When I am using cat, head or tail command on ...
1
vote
2answers
151 views

Renaming folders based on a dictionary in form of a CSV file?

I will describe the problem quite verbose, as I have learned that only talking about a specific sub-problem I have identified leads to missed oppurtunities ... So I will put a tl;dr up front: There is ...
4
votes
5answers
6k views

extract the 4th column from a csv file using unix command

I have a csv file formatted as below. "col1","col2","col3","col4" "1","text1","<p>big html text</p> ","4th column" "2","text2","<p>big2 html2 text2</p> ...
2
votes
2answers
3k views

Convert csv to HTML table

I have a Medical.csv file with rows of following format, field: 'participation.type', displayName: 'program_type', type: 'String',path:'participation' field: 'participation.program', ...
5
votes
5answers
6k views

Parsing a delimited text file in bash as command arguments

I have a text file split up like so: field1,field2,field3 xield1,xield2,xield3 dield1,dield2,dield3 gield1,gield2,gield3 Each of these columns will be a parameter to a program, and I would like ...
2
votes
2answers
382 views

Print some fields of /etc/passwd

I have a question concerning the usage of wild cards in Linux. Here I have some lines from a long list of lines in the /etc/passwd -file. root:x:0:0:root:/root:/bin/bash ...
0
votes
0answers
107 views

Error while Converting Excel to CSV using xls2csv

I am using xls2csv to convert a xls file into a regular .csv file. However, while using the command as xls2csv -c\, fromfile.xls > tofile.csv .I am getting the following error. Do not know what ...