Files containing data arrange in a table, often with commas (hence Comma Separated Values), to separate columns. Rows are separated by newlines (but not all newlines are row separators as fields can be quoted to contain the separator newlines. Use this tag for full-fledged CSV data not the simpler ...

learn more… | top users | synonyms

1
vote
2answers
153 views

Replace only first instance of a character

for example I have scv file which looks like a1, b1, c1, d1 a2, b2, c2, d2 a3, b3, c3, d3 What I want to do is to replace the first comma , with the semicolon ;. The position of first comma can be ...
0
votes
0answers
13 views

How do I convert a CSV file to a map or ped file which is compatible with PLink?

Working with large amounts of data in a CSV file and must convert them to map or ped to allow compatibility with PLink. How do I change the type of file?
2
votes
1answer
61 views

parsing several fields of a csv with gawk on solaris

I have a generated csv that has the following format: unixtime;host_name;ip_adress;description;2;0;1 I have been trying to parse this csv to do the following: change field 1 from unix timestamp ...
4
votes
4answers
595 views

Parse a txt file based on a specific delimeter and then tranform it into a CSV file

I have the following file named OpenSimStats.txt: TestreportsRootAgentCount=0agent(s) TestreportsChildAgentCount=0childagent(s) TestreportsGCReportedMemory=10MB(Global) TestreportsTotalObjectsCount=...
1
vote
2answers
73 views

A script to take data from csv file and perform related operations in a database

I am new to the world of shell scripting and don't have much idea of shell scripting . My work requires me to write a script which takes phone numbers from a list of records stored in an CSV file . ...
0
votes
3answers
45 views

Print/export csv file into array

I have the following data in a very large csv: "sortorder","name","description" "1","B.1","Boiler room" "2","1.1","First office" "3","1.2","Second office" etc... Which I would like to end up in ...
1
vote
2answers
93 views

Merging contents of multiple .csv files into single .csv file

I want to write a script that merges contents of several .csv files in one .csv file, i.e appends columns of all other files to the columns of first file. I had tried doing so using a "for" loop but ...
4
votes
2answers
42 views

Is it possible to print an iterated range of arguments via cli from a csv? (see example)

I have hundreds of logs spread across a few thousand VM's and I'm trying to create a few scripts to quickly scan logs and I have figured out the majority of the presentation and wanted to know if ...
0
votes
3answers
38 views

Combining several text files into a single csv [closed]

Input files: File: Article1.txt: paragraph1 It is a long established fact that a reader will...... paragraph2 It is a long established fact that a reader will...... paragraph3 It is a long ...
0
votes
3answers
62 views

How can I retrieve and compare two values from a file? [closed]

How can I retrieve and compare two values from the first two lines of a file? I have the following text: 05-24-2016, 2:59:32,0,0 05-24-2016, 2:59:37,0,0 05-24-2016, 2:59:42,0,0 05-24-2016, 2:59:47,0,...
3
votes
2answers
107 views

Question about concatenation instructions

I have a semicolon-delimited csv file and a Korn shell script that do some elaboration. I'm searching to understand what these two instructions do in Korn shell: num_rec=`cat test.csv |wc -l|tr -d " "...
1
vote
2answers
78 views

Removing spaces from the fields in a semicolon-delimited file using shell script

I have a csv file like this: 03139; 5; IT1234978208; 20150930 ; CTZ 13/31.12.15; 03137; 6; IT1234978206; 20151015 ; CTZ 13/31.11.18; .... .... 03134; 8; IT1234974406; ...
0
votes
1answer
62 views

Matching 3 or more words from fields in separate csv files

I have two csv files: This is csv1: 11, The Sun Still Shines in Reading, 64312, 464566 13, You and Me Together Again London, 564564, 131355 12, What's the Story Now Your Gone, 4545646, 1124545 17, ...
0
votes
1answer
61 views

Match words in array line by line to a seperate file

Here is a sample of my input data: $9.99,Titan the Power,www.example.com,565654 $15.99,Conan The Barbarian,www.sure.com,565438 $1.99,Julia Cesar,www.bfg.com,69722 I have written this code so ...
1
vote
2answers
29 views

Can python extract org-mode tables from org documents?

I'm trying to write some python code that reads specified tables in org mode documents. For example, I have a file ~/foo.org $ cat ~/foo.org #+Title: Example Org Document * Section One Proin quam ...
2
votes
3answers
51 views

Bash read with delimiter past newlines

I have a csv for which I have to extract a specific field. The csv is pipe-delimited (|), with double quotes (") to protect text fields and (this is the catch) newlines in some of the text. eg. "aaa"...
24
votes
2answers
1k views

How to merge two text files into one file without using cat or sed [duplicate]

I have two files File1: a b c File2: 1 2 3 now I need to combine them to one csv file a;1 b;2 c;3 As the files are really huge, I would rather not use cat and sed to process the second file. ...
3
votes
1answer
38 views

If there are more than x number of pipes in a csv line, then delete the 2nd instance

I have a csv file which is supposed to contain 4 columns of data which include a product number, a title, a url and a price. Each column is separated by a | delimiter (this has to be maintained, there ...
2
votes
3answers
52 views

linux + ssh + perform two tasks in one ssh and print to CSV

the following command print the two values in one ssh shut ssh test01 "hostname && cat /etc/sysconfig/rhn/rhnsd " test01 INTERVAL=240 but how to change the syntax in order to print the ...
3
votes
1answer
52 views

Copy csv data while simultaneously filling serial column

I've created my first postgres table and i'm trying to work out how to copy a .csv file into the table, whilst simultaneously generating a serial number number in column 1 (ID) which starts at number ...
2
votes
3answers
72 views

Splitting file based on last digit of first column in UNIX

I have a file sales_$date.csv. I want to split it into 10 files based on the last digit of first column (ITEM). so in reality file would be split in 10 files as sales_$date-01.csv, sales_$date-02.csv,....
0
votes
4answers
101 views

How do I remove all \r\n from a file, but preserve \n

I have a CSV with unix line endings, but some of the string values have windows line endings in them: date,notes\n 2014-01-01,"Blah Blah Blah"\n 2014-01-02,"Two things:\r\n - first thing\r\n - ...
0
votes
1answer
75 views

Extract data from csv

friends! I have a HUGE csv file and I need extract rows with headers where column called "data.status" equals "complete", then delete empty columns from it and write to another csv file. Can I achieve ...
1
vote
0answers
64 views

How to make excel interpret Chinese characters in CSV file

I am sending a report from a unix shell script that contains chinese characters. When I open the CSV report in plaintext editor like textmate I can see the chinese characters but when I open CSV ...
0
votes
2answers
88 views

Compare a file with a column of another file

I have a file File1 with data as aaa bbb And another file File2 with data as: 2,aaa,234 w,bbb,589 4,ccc,675 I need to compare the File1 data with column2 data of File2 and print the matching ...
1
vote
1answer
32 views

Calculate hits within a timespan from log file [closed]

I need to use UNIX utilities that implement out-of-core divide-and-conquer algorithms so to decrease the amount of virtual memory used (cap of 1 mb). I have a large data set that is an apache log file....
1
vote
0answers
44 views

CSV copyscript not copying 5 files out of 4000+

I was provided a great bit of code for searching and copying files listed in a csv from a directory and its subfolders in an answer to my previous question. However, some files are not copied. The ...
0
votes
1answer
131 views

text to xls file creation

I have 4 text files in my unix system want to put data into a xls file and send it via mail. text1.txt: 100 150 130 120 110 text2.txt: 200 230 240 250 260 .. .. .. so on ... I want to have ...
-3
votes
2answers
30 views

How can I reformat MAC address and IP pairs, ensuring each byte of the MAC has a leading zero?

I've got a table like this: 10.6.6.12 0:50:56:9b:a6:2 10.6.6.13 fe:9:31:b6:71:1d I must have something like this: 10.6.6.12,00:50:56:9b:a6:02 10.6.6.13,fe:09:31:b6:71:1d A 0 is always added ...
4
votes
3answers
135 views

Delete consecutive lines in CSV with duplicate values in one field, but keep the last line

I have a long CSV file with two columns, which includes runs of consecutive duplicates like this: ... 1500,1533 1554,1678 1554,1703 1554,1728 1593,1766 ... I need to delete all of these duplicates ...
1
vote
2answers
42 views

I would like join two csv files

I would like join two csv files: file1: 19/02/2016 22:00;16.70 19/02/2016 22:30;16.80 19/02/2016 23:00;16.80 19/02/2016 23:30;16.70 file2: 19/02/2016 22:00;39.50 19/02/2016 22:30;41.00 19/02/2016 ...
0
votes
1answer
30 views

Parsing a delimited in ksh as command argument

My system generates a flat file with 7 arguments : Field1,Field2,Field3,Field4,Field5,Field6,Field7 Field1,Field2,Field3,Field4,Field5,Field6,Field7 Field1,Field2,Field3,Field4,Field5,Field6,Field7 ...
1
vote
4answers
192 views

Split comma separated list ignoring commas within matching {}

I want to split a csv, but ignore any commas within matching curly brace groups, and loop through each of the list members. The code below works great but does not consider commas within curly brace ...
0
votes
3answers
83 views

How to find specific csv files and extract their second row into a single csv file

I'm trying to find out how to filter specific csv files by name and then extract their second row into a single csv file. I'm not sure whether it is a good idea to find files and then pipe it with ...
1
vote
0answers
222 views

How to redirect linux command outputs to specific row and column in csv?

I'm working on Unix Build post documentation process, I have a CSV file to be filled in and uploaded for every new build, to ensure the servers are build as per the original request. The CSV file ...
4
votes
5answers
254 views

How do I print all lines of a file with duplicate values in a certain column

Given a file like this: 1,768,12,46576457,7898 1,123,435,134,146 2,345,6756856,12312,1311 5,234,567465,12341,1341 1,3245,4356345,2442,13 9,423,2342,121,463 9,989,342,121,1212 I would like to list ...
6
votes
7answers
785 views

Keep only the lines containing exact number of delimiters

I have a huge csv file with 10 fields separated by commas. Unfortunately, some lines are malformed and do not contain exactly 10 commas (what causes some problems when I want to read the file into R). ...
-4
votes
1answer
54 views

How to transform row to collumn in csv file?

2015-12-2119:30:00;ManagedElement=1,Equipment=1,Subrack=1 ;pmEs ;0 I need this result 2015-12-2119:30:00;ManagedElement=1,Equipment=1,Subrack=1;pmEs;0 Extra data from comment: 2015-12-21 19:30:...
5
votes
7answers
869 views

Remove lines from tab-delimited file with missing values

I have a large (~900MB) tab-delimited text file that I will be processing in a downstream program. I need to delete any line with a missing value. The correct number of columns are on each line (so ...
2
votes
0answers
42 views

Command like `column -t` but encoding-type agnostic? [closed]

Essentially, I am looking for an encoding-type agnostic version of the column -t command in linux. I need to convert delimited files to flat and then write to an outfile. The specific delimiter ...
1
vote
4answers
314 views

Modify an SVG file based on information in a CSV file

I've got a SVG file of the map of the US and I want to add colors to it. Because it would take me to much time, I thought I could use a bash script. I have also another file containing the states and ...
3
votes
1answer
279 views

Extract specific columns from text file

There is a text file, with aboot 10000+ lines, in the format: field1 field2 field3 field4 field5 ... fieldX It needs to be converted to look like: field1,field2,field4,field3,field5 to fieldX The ...
0
votes
0answers
37 views

Assign Tracking # to a CSV file from a Master List TXT File

I have a text file that contains a list of tracking numbers (MasterList.txt). I have another file, Orders.csv, file with 3 columns. Row 1 is the headers. The last column is the 'Tracking Number' ...
1
vote
4answers
196 views

Execute Linux command only if column in CSV file has a value greater than 1

I am trying to figure out a way to execute a Linux command only if a specific column in my CSV file has a value greater than 1. Sample CSV file: Item,Quantity Pen,1 Pencil,1 Stapler,3 ...
1
vote
1answer
181 views

get mysql count of id in shell script

I am trying to get count of column user_Id using count(user_Id) from mysql as follows: count=$(mysql -uroot -proot csv_imports -e "select count(user_Id) from test_data where user_Id=\"12345\";") I ...
-1
votes
1answer
31 views

Reformat text contents into a CSV

I have a text file that I wish to turn into a CSV. The pattern of its contents are as follows: 1 mm/dd/yyyy LastName, FirstName MiddleName ID-NUMBER-HERE GENDER 2 mm/dd/yyyy LastName, FirstName ...
2
votes
3answers
102 views

Addition of values in second columns if the first column entry is same in UNIX

I am trying to aggregate a file containing the following data in UNIX. I need to add the amounts if the key is same. Key,amount,date,Time abc-xyz-12234,45,15-08-91,23:00 pqr-vgh-5241,15,15-08-91,21:...
0
votes
1answer
50 views

AWK Help ending standalone script

I started to do this.. to swap fields in a csv file but I don't get to finish the script. Any help? What I want to do is to concatenate fields from 2 to 7 and group them by the first field. Fields 1, ...
1
vote
0answers
24 views

exchange a column of many many csv files using awk [duplicate]

There is a similar post, but this one is slightly different. I have a csv file in which the second column contains some alphanumeric data. I need to delete this entire column and replace it with a new ...
1
vote
4answers
243 views

Changing column of multiple csv files

I have a csv file called scenario1.csv in which the second column has column names like "0-4 years high risk", "65+ years first responder" etc. There are 20 such values. The 21st row, 2nd column has ...