Files containing data arrange in a table, often with commas (hence Comma Separated Values), to separate columns. Rows are separated by newlines (but not all newlines are row separators as fields can be quoted to contain the separator newlines. Use this tag for full-fledged CSV data not the simpler ...

learn more… | top users | synonyms

4
votes
2answers
40 views

Is it possible to print an iterated range of arguments via cli from a csv? (see example)

I have hundreds of logs spread across a few thousand VM's and I'm trying to create a few scripts to quickly scan logs and I have figured out the majority of the presentation and wanted to know if ...
0
votes
3answers
38 views

Combining several text files into a single csv [closed]

Input files: File: Article1.txt: paragraph1 It is a long established fact that a reader will...... paragraph2 It is a long established fact that a reader will...... paragraph3 It is a long ...
0
votes
3answers
61 views

How can I retrieve and compare two values from a file? [closed]

How can I retrieve and compare two values from the first two lines of a file? I have the following text: 05-24-2016, 2:59:32,0,0 05-24-2016, 2:59:37,0,0 05-24-2016, 2:59:42,0,0 05-24-2016, 2:59:47,0,...
3
votes
2answers
106 views

Question about concatenation instructions

I have a semicolon-delimited csv file and a Korn shell script that do some elaboration. I'm searching to understand what these two instructions do in Korn shell: num_rec=`cat test.csv |wc -l|tr -d " "...
1
vote
2answers
72 views

Removing spaces from the fields in a semicolon-delimited file using shell script

I have a csv file like this: 03139; 5; IT1234978208; 20150930 ; CTZ 13/31.12.15; 03137; 6; IT1234978206; 20151015 ; CTZ 13/31.11.18; .... .... 03134; 8; IT1234974406; ...
0
votes
1answer
62 views

Matching 3 or more words from fields in separate csv files

I have two csv files: This is csv1: 11, The Sun Still Shines in Reading, 64312, 464566 13, You and Me Together Again London, 564564, 131355 12, What's the Story Now Your Gone, 4545646, 1124545 17, ...
0
votes
1answer
60 views

Match words in array line by line to a seperate file

Here is a sample of my input data: $9.99,Titan the Power,www.example.com,565654 $15.99,Conan The Barbarian,www.sure.com,565438 $1.99,Julia Cesar,www.bfg.com,69722 I have written this code so ...
1
vote
2answers
28 views

Can python extract org-mode tables from org documents?

I'm trying to write some python code that reads specified tables in org mode documents. For example, I have a file ~/foo.org $ cat ~/foo.org #+Title: Example Org Document * Section One Proin quam ...
2
votes
3answers
47 views

Bash read with delimiter past newlines

I have a csv for which I have to extract a specific field. The csv is pipe-delimited (|), with double quotes (") to protect text fields and (this is the catch) newlines in some of the text. eg. "aaa"...
24
votes
2answers
1k views

How to merge two text files into one file without using cat or sed [duplicate]

I have two files File1: a b c File2: 1 2 3 now I need to combine them to one csv file a;1 b;2 c;3 As the files are really huge, I would rather not use cat and sed to process the second file. ...
3
votes
1answer
37 views

If there are more than x number of pipes in a csv line, then delete the 2nd instance

I have a csv file which is supposed to contain 4 columns of data which include a product number, a title, a url and a price. Each column is separated by a | delimiter (this has to be maintained, there ...
2
votes
3answers
47 views

linux + ssh + perform two tasks in one ssh and print to CSV

the following command print the two values in one ssh shut ssh test01 "hostname && cat /etc/sysconfig/rhn/rhnsd " test01 INTERVAL=240 but how to change the syntax in order to print the ...
3
votes
1answer
41 views

Copy csv data while simultaneously filling serial column

I've created my first postgres table and i'm trying to work out how to copy a .csv file into the table, whilst simultaneously generating a serial number number in column 1 (ID) which starts at number ...
2
votes
3answers
69 views

Splitting file based on last digit of first column in UNIX

I have a file sales_$date.csv. I want to split it into 10 files based on the last digit of first column (ITEM). so in reality file would be split in 10 files as sales_$date-01.csv, sales_$date-02.csv,....
0
votes
4answers
90 views

How do I remove all \r\n from a file, but preserve \n

I have a CSV with unix line endings, but some of the string values have windows line endings in them: date,notes\n 2014-01-01,"Blah Blah Blah"\n 2014-01-02,"Two things:\r\n - first thing\r\n - ...
0
votes
1answer
68 views

Extract data from csv

friends! I have a HUGE csv file and I need extract rows with headers where column called "data.status" equals "complete", then delete empty columns from it and write to another csv file. Can I achieve ...
1
vote
0answers
55 views

How to make excel interpret Chinese characters in CSV file

I am sending a report from a unix shell script that contains chinese characters. When I open the CSV report in plaintext editor like textmate I can see the chinese characters but when I open CSV ...
0
votes
2answers
73 views

Compare a file with a column of another file

I have a file File1 with data as aaa bbb And another file File2 with data as: 2,aaa,234 w,bbb,589 4,ccc,675 I need to compare the File1 data with column2 data of File2 and print the matching ...
1
vote
1answer
32 views

Calculate hits within a timespan from log file [closed]

I need to use UNIX utilities that implement out-of-core divide-and-conquer algorithms so to decrease the amount of virtual memory used (cap of 1 mb). I have a large data set that is an apache log file....
1
vote
0answers
43 views

CSV copyscript not copying 5 files out of 4000+

I was provided a great bit of code for searching and copying files listed in a csv from a directory and its subfolders in an answer to my previous question. However, some files are not copied. The ...
0
votes
1answer
106 views

text to xls file creation

I have 4 text files in my unix system want to put data into a xls file and send it via mail. text1.txt: 100 150 130 120 110 text2.txt: 200 230 240 250 260 .. .. .. so on ... I want to have ...
-3
votes
2answers
30 views

How can I reformat MAC address and IP pairs, ensuring each byte of the MAC has a leading zero?

I've got a table like this: 10.6.6.12 0:50:56:9b:a6:2 10.6.6.13 fe:9:31:b6:71:1d I must have something like this: 10.6.6.12,00:50:56:9b:a6:02 10.6.6.13,fe:09:31:b6:71:1d A 0 is always added ...
4
votes
3answers
126 views

Delete consecutive lines in CSV with duplicate values in one field, but keep the last line

I have a long CSV file with two columns, which includes runs of consecutive duplicates like this: ... 1500,1533 1554,1678 1554,1703 1554,1728 1593,1766 ... I need to delete all of these duplicates ...
1
vote
2answers
42 views

I would like join two csv files

I would like join two csv files: file1: 19/02/2016 22:00;16.70 19/02/2016 22:30;16.80 19/02/2016 23:00;16.80 19/02/2016 23:30;16.70 file2: 19/02/2016 22:00;39.50 19/02/2016 22:30;41.00 19/02/2016 ...
0
votes
1answer
29 views

Parsing a delimited in ksh as command argument

My system generates a flat file with 7 arguments : Field1,Field2,Field3,Field4,Field5,Field6,Field7 Field1,Field2,Field3,Field4,Field5,Field6,Field7 Field1,Field2,Field3,Field4,Field5,Field6,Field7 ...
1
vote
4answers
173 views

Split comma separated list ignoring commas within matching {}

I want to split a csv, but ignore any commas within matching curly brace groups, and loop through each of the list members. The code below works great but does not consider commas within curly brace ...
0
votes
3answers
75 views

How to find specific csv files and extract their second row into a single csv file

I'm trying to find out how to filter specific csv files by name and then extract their second row into a single csv file. I'm not sure whether it is a good idea to find files and then pipe it with ...
1
vote
0answers
196 views

How to redirect linux command outputs to specific row and column in csv?

I'm working on Unix Build post documentation process, I have a CSV file to be filled in and uploaded for every new build, to ensure the servers are build as per the original request. The CSV file ...
4
votes
5answers
218 views

How do I print all lines of a file with duplicate values in a certain column

Given a file like this: 1,768,12,46576457,7898 1,123,435,134,146 2,345,6756856,12312,1311 5,234,567465,12341,1341 1,3245,4356345,2442,13 9,423,2342,121,463 9,989,342,121,1212 I would like to list ...
6
votes
7answers
704 views

Keep only the lines containing exact number of delimiters

I have a huge csv file with 10 fields separated by commas. Unfortunately, some lines are malformed and do not contain exactly 10 commas (what causes some problems when I want to read the file into R). ...
-4
votes
1answer
54 views

How to transform row to collumn in csv file?

2015-12-2119:30:00;ManagedElement=1,Equipment=1,Subrack=1 ;pmEs ;0 I need this result 2015-12-2119:30:00;ManagedElement=1,Equipment=1,Subrack=1;pmEs;0 Extra data from comment: 2015-12-21 19:30:...
5
votes
7answers
789 views

Remove lines from tab-delimited file with missing values

I have a large (~900MB) tab-delimited text file that I will be processing in a downstream program. I need to delete any line with a missing value. The correct number of columns are on each line (so ...
2
votes
0answers
42 views

Command like `column -t` but encoding-type agnostic? [closed]

Essentially, I am looking for an encoding-type agnostic version of the column -t command in linux. I need to convert delimited files to flat and then write to an outfile. The specific delimiter ...
1
vote
4answers
299 views

Modify an SVG file based on information in a CSV file

I've got a SVG file of the map of the US and I want to add colors to it. Because it would take me to much time, I thought I could use a bash script. I have also another file containing the states and ...
3
votes
1answer
257 views

Extract specific columns from text file

There is a text file, with aboot 10000+ lines, in the format: field1 field2 field3 field4 field5 ... fieldX It needs to be converted to look like: field1,field2,field4,field3,field5 to fieldX The ...
0
votes
0answers
36 views

Assign Tracking # to a CSV file from a Master List TXT File

I have a text file that contains a list of tracking numbers (MasterList.txt). I have another file, Orders.csv, file with 3 columns. Row 1 is the headers. The last column is the 'Tracking Number' ...
1
vote
4answers
183 views

Execute Linux command only if column in CSV file has a value greater than 1

I am trying to figure out a way to execute a Linux command only if a specific column in my CSV file has a value greater than 1. Sample CSV file: Item,Quantity Pen,1 Pencil,1 Stapler,3 ...
1
vote
1answer
146 views

get mysql count of id in shell script

I am trying to get count of column user_Id using count(user_Id) from mysql as follows: count=$(mysql -uroot -proot csv_imports -e "select count(user_Id) from test_data where user_Id=\"12345\";") I ...
-1
votes
1answer
30 views

Reformat text contents into a CSV

I have a text file that I wish to turn into a CSV. The pattern of its contents are as follows: 1 mm/dd/yyyy LastName, FirstName MiddleName ID-NUMBER-HERE GENDER 2 mm/dd/yyyy LastName, FirstName ...
2
votes
3answers
89 views

Addition of values in second columns if the first column entry is same in UNIX

I am trying to aggregate a file containing the following data in UNIX. I need to add the amounts if the key is same. Key,amount,date,Time abc-xyz-12234,45,15-08-91,23:00 pqr-vgh-5241,15,15-08-91,21:...
0
votes
1answer
50 views

AWK Help ending standalone script

I started to do this.. to swap fields in a csv file but I don't get to finish the script. Any help? What I want to do is to concatenate fields from 2 to 7 and group them by the first field. Fields 1, ...
1
vote
0answers
24 views

exchange a column of many many csv files using awk [duplicate]

There is a similar post, but this one is slightly different. I have a csv file in which the second column contains some alphanumeric data. I need to delete this entire column and replace it with a new ...
1
vote
4answers
235 views

Changing column of multiple csv files

I have a csv file called scenario1.csv in which the second column has column names like "0-4 years high risk", "65+ years first responder" etc. There are 20 such values. The 21st row, 2nd column has ...
1
vote
2answers
162 views

How to convert my text files (each containing a username+password pair) into CSV in order to import them to KeePass via bash(cygwin)

I'd like to use Keepass (win10x64). The problem is that, I have my passwords gathered all these years like this: each username/password pair in a separate text file, having as filename the domain ...
3
votes
3answers
247 views

awk when both delimiter and quotes are used for a field

I have a file in the following format: field1|field2|field3 field1|"field2|field2"|field3 Notice the second row contains double quotes. The string within the double quotes belongs to field 2. How ...
0
votes
1answer
99 views

Single read filter csv based on 1st column values

I have a gigantic csv file that is around a GB in size. The csv's 1st column contains around 300 unique integer values. I want to process the csv once and save the output to different files based on ...
1
vote
2answers
58 views

Stack data from multiple delimited files into one, with variable columns

I'm new to scripting. Got struck with a file merge issue in Unix. Was looking for some direction and stumbled upon this site. I saw many great posts and replies but couldn't find a solution to my ...
2
votes
1answer
76 views

Processing table with comma separated values in different columns

I'm trying to do the following for a while but without success. The data I received has comma separated values in each separate columns. The first value in column 6 before the comma is always related ...
2
votes
1answer
148 views

Adding two columns together in CSV and outputting to new CSV file

I have a problem I have been trying to figure out: We have a stock file CSV, which contains the stock in multiple locations. The csv looks like this: stock_no,primary,secondary,tertiary,cstock,...
0
votes
3answers
37 views

List users who have the maximum number of processes running with maximum cpu usage

starting off with the command ps -eo user,pcpu,pid,args | sort -rnk 3 | head -30 How do i parse the output of this command into a csv file with users having most processess and consuming cpu usage?...