Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [csv-simple]

Text files containing simple comma separated values. Use this for data that is a subset of real CSV without newlines in (quoted) field content, that can be handled with line oriented tools without the need for a real parser.

0
votes
2answers
93 views

SED not removing last double quote from every line

I have a CSV that was exported with double quotes around every header and value and I need them gone but without removing double quotes that might actually exist as values. For example: "HEADER1","...
-1
votes
3answers
33 views

How to extract lines when two columns strings are not equal

I have a CSV file that looks like this format: text1,text2,string1,string2 text3,text3,string3,string2 text4,text5,string1,string2 text6,text6,string6,string7 I want to extract rows when column1 and ...
2
votes
5answers
703 views

awk: split column, print left side of delimiter and paste in the middle of csv

Input file: AAA, BBB:XXX, CCC, DDD, EEE, FFF, GGG, HHH Output should be: AAA, BBB, BBB:XXX, CCC, DDD, EEE, FFF, GGG, HHH I thought about something like: awk -F, '{n=split($2,a,":"); a[n]} {$2=$NF"...
0
votes
2answers
70 views

Append to second column

There is a csv file ip,time,name 1.1.1.1,2018-08-31 10:01:03,postgres 2.1.1.1,2018-08-31 12:51:13,oracle 3.1.1.1,2018-08-31 14:32:53,sql I need output as below ip,time,name 1.1.1.1,2018-08-31 10:...
0
votes
3answers
107 views

Replace multiple variable values in a file with given input values (from a file)

If i have two input file with contents like below: How i can replace network_name (eg:P1MSVmgmtvM replace with /rest/ethernet-networks/2bf8bc44-5b31-4e6c-99ed-d68b1e9acf9e). File1: (input file) ...
0
votes
1answer
43 views

print each Network set in separate same group

Currently my csv input looks like below: network_name,network_set P1MSVmgmtvM,Data_NetworkSet_A P1MSVvMotion,Data_NetworkSet_A P2MSVmgmtvM,Data_NetworkSet_B P2MSVvMotion,Data_NetworkSet_B ...
1
vote
3answers
309 views

remove duplicated pattern/entries within each field in CSV file

How do I remove duplicated entries within each separate fields with below sample as data. 0x,9.4,,,#0,#UNIX#unix,#cli#L#فا#0#فا#0#L#SE#Cli#SE,#فارسی#فارسی#۱#1#١#1,bsh,#V & v expected output(...
0
votes
3answers
597 views

Using AWK To Extract Numbers From .CSV File

I have a .CSV file that I need to extract numbers from. Example of file: #File of numbers 1,2,3,4,5 6,7,8,9,10 I want to use awk (or another utility) to extract the first number and second number ...
1
vote
1answer
2k views

How to insert CSV data into an SQLite table via a shell pipe?

I have written a program that outputs the result to the standard output in strict pure CSV form (every line represents a single record and contain the same set of comma-separated fields, fields only ...
3
votes
1answer
52 views

Why is column adding a newline in the middle of my row where one is not present in the original data?

I'm analyzing some packetfilter logs and wanted to make a nice table of some output, which normally works fine when I use column -t. I can't use a tab as my output field separator (OFS) in this case ...
0
votes
2answers
38 views

Checking if a text file has all entries and is formatted correctly

I have a file that is like this 0001;4;5;30;33;41;52 0002;9;37;39;41;43;49 0003;10;11;29;30;36;47 0004;1;5;6;27;42;59 0005;1;2;6;16;19;46 0006;7;13;19;22;40;47 0007;3;5;20;21;38;56 ... etc The first ...
0
votes
1answer
490 views

How can I replace Nth delimiter in a csv file?

I have a comma seperated csv file as a sample like: 1,92345,92345,Dear user, this is your amount , 2016-10-10 2,92345,92345,Dear user, this is your amount , 2016-10-09 I need to replace 4th comma , ...
5
votes
7answers
771 views

How to remove duplicate value in a tab-delimited text file

I have a tab delimited column text like below A B1 B1 C1 B B2 D2 C C12 C13 C13 D D3 D5 D9 G F2 F2 how could I convert the above table ...
0
votes
2answers
146 views

Pivoting Dynamic row data to columns wrt unique values in another column

I'm stuck with a particular requirement which requires the conversion of dynamic row data to column fields. Suppose I am having a file "output.csv" with the contents like: TimeStamp,Block,...
5
votes
1answer
3k views

Print each field of CSV on newline without knowing number of fields

I was playing with IFS today and created a quick text file with a list of numbers separated by commas on 1 line. 1,2,3,4,5 I then tried to write a script to print each number on a newline. I was ...
-1
votes
2answers
538 views

Add a header field, and the file's name at the end of the data lines, for all the csv files in a folder [closed]

I want to add filename (without extension) at the end for all the lines for all the csv files in a folder. All the files are having the same header. Let's say I have two files a.csv and b.csv in a ...
2
votes
5answers
1k views

Use AWK to split substring by last n characters in to a new column

I have a large .csv file where I need to split a specific column by string length. I'm trying to take the last 6 characters of column 2 and move them into a new column. Current: 3102017,90131112,0,...
3
votes
3answers
3k views

Count the number of occurrences of a column value in a TSV file with AWK

I have a TSV tab-separated file with 3 cols: ID\tTEXT\tTYPE To print the TYPE column I do cat /dataset.csv | awk -F $'\t' '{print $3}' Those values are an enumeration of values like {CLASS_A,...
22
votes
12answers
12k views

Converting CSV to TSV

I have a number of large CSV files and would like them in TSV (tab separated format). The complication is that there are commas in the fields of the CSV file, eg: A,,C,"D,E,F","G",I,"K,L,M",Z ...
1
vote
2answers
112 views

text processing : extract parts of a file and write them into a single line

I have several files formatted like this: ... <title> Field1 : DATA_FIELD_1</title> ... <i class="blablabla"></i> <b>Field2 : </b> <span>DATA_FIELD_2</...
0
votes
2answers
991 views

Calculate sum of a column using shellscript

I have a csv file as text.csv and want to calculate the total salary of all employees. The data in it is as follows: Id,name,salary,designation 12,joe,thomas,253637,IT,admin 25,miacheal,364829,...
0
votes
1answer
187 views

appending three CSV files [duplicate]

I have three CSV files which have the same same row count. 1.csv 1,2,3 4,5,6 7,8,9 2.csv 4 5 6 3.csv 8 9 10 After combining I want the result as result.csv 1,2,3,4,8 4,5,6,5,9 7,8,9,6,10 So I ...
1
vote
3answers
210 views

merge multiple lines of data having one common field

I have a large file with 9+ rows of data seperates with semi-colon(;) and I want to merge data in column 3 (seperating with ,) the lines where data in column 5 matches. The data is held on a Linux ...
1
vote
2answers
74 views

How to convert uA/mA to uA in each lines of csv file using c shell?

I have a *.csv file as bellow. What I want to do is converting all units to one unit (uA). Anyone can help to do this in c shell??
3
votes
4answers
5k views

How to check for null fields in CSV file?

My CSV file will look like this (pipe separated): apple|banana|pear||grapefruit lemon|lime|damson|jackfruit |tangerine|nectarine|plum apricot|orange|pineapple|coconut| (Assume that any sort of ...
4
votes
4answers
201 views

Double quote value assignments stored in a CSV?

I have a file that contains text as follows: dt=2016-06-30,path=path1,site=US,mobile=1 dt=2016-06-21,path=path2,site=UK,mobile=0 I want to convert it to text with double-quoted values in the key-...
2
votes
1answer
408 views

Command to “fill down” columns in text file, a la the Excel fill down function

I have a text file with rows, and columns within those rows. I want to, essentially, replicate the Excel "fill down" function. In other words, if there is a blank "cell" on a line, it will look to the ...
3
votes
1answer
21 views

Update table header regarding file name

I have several files named has follow (I just give 2 names as examples)(note: they have a semi colon ; field separator): component1_component2_component3.csv e.g. almourol_gfdl_85.csv ...
0
votes
1answer
31 views

Append filenames to csv-file if matching id

I have two files file1 159211;exampleimage002.JPG 159212;example.jpg 159212;example with space.jpg ... file2 13;"OVET";"O1";"OVET/ULKO-OVET";159212;"JELDWEN BASIC 0015 9*21 VASEN";"KPL";1;345;1;0;"...
1
vote
2answers
32 views

Data extraction problem based on upper case words

I have a data file in the following format: source.dat (Example for illustration purposes.) ALBANIA Duck 1000 BELGIUM CHARLEROI Donald Duck 10234 CYPRUS J. Mickey 567 I'm looking for a Linux script ...
0
votes
3answers
55 views

Change CSV records sequentially

I want to replace the text sequentially in a file. The file's contents are: 1,01-JUN-2016,ABC 2,01-JUN-2016,ABC 3,01-JUN-2016,ABC 4,01-JUN-2016,ABC 5,01-JUN-2016,ABC 6,02-JUN-2016,ABC 7,02-JUN-2016,...
1
vote
2answers
1k views

Convert command output to csv, with time stamp

I want to test a speed connection with terminal command: ./speedtest-cli And it returns This: Retrieving speedtest.net configuration... Retrieving speedtest.net server list... Testing from Moscow, ...
1
vote
1answer
681 views

AWK, Sum of category

I have tons of CSV files with similar content. The values are usually comma separated and they look like this. product_a, domestic, 500 product_a, abroad, 15 product_b, domestic, 313 ...
3
votes
1answer
348 views

Create table with frequency of unique names retrieved from multiple .csv files

I have 32 CSV files containing fetched information from a database. I need to make a frequency table in TSV/CSV format, where the names of the rows are the name of each file, and the names of the ...
1
vote
1answer
838 views

I need to merge multiple .csv files together side by side with unequal number of rows

I have anywhere from 3-5 .csv files and need to be able to merge them together while keeping everything in its respective columns below is an easy example with files with varying number of rows. file1 ...
1
vote
4answers
81 views

Remove any occurrence of '-' from a field in shell script

below is my csv file,I want to remove all occurrences of - from retailer_id field and create new csv. >IPAY_USER_ID,RETAILER_ID,CUST_FIRST_NAME,CUST_LAST_NAME,CUST_MIDDLE_NAME,ACTIVATION_ACTOR_ID,...
3
votes
3answers
1k views

Search a directory recursively for files listed in a csv, and copy them to another location

I have a directory (let's call it "Movies") which contains many files and folders. I have a long list of file names in a .csv file (around 4000 entries) which refer to files which are located ...
0
votes
1answer
377 views

Add column to a CSV file as a function of other two columns, one from other file

Here is the situation: a.csv Contains values from some sensors captured every 1 second, the last column is the epoch (they actually looks like 1453998477.044).(*s are the sensors values). Example: ...
-4
votes
1answer
85 views

How to transform row to collumn in csv file?

2015-12-2119:30:00;ManagedElement=1,Equipment=1,Subrack=1 ;pmEs ;0 I need this result 2015-12-2119:30:00;ManagedElement=1,Equipment=1,Subrack=1;pmEs;0 Extra data from comment: 2015-12-21 19:30:...
4
votes
5answers
710 views

Count unique associated values in awk (or perl)

I've already found "How to print incremental count of occurrences of unique values in column 1", which is similar to my question, but the answer isn't sufficient for my purposes. First let me just ...
0
votes
2answers
133 views

Transform row of integer into multi-column output

I am trying to get the input of a file to generate me a list with rows and columns of that numbers in sorted order. This will be use to generate a printable document later, so i am able to replace the ...
0
votes
3answers
551 views

Removing white space using sed but skipping Date-Time stamp

I wanted to remove white space from a CSV file, which I am able to do using s/\ //g, but at the same time I want to avoid removing the space between fields which has timestamps e.g. "06-JAN-15 13:20:...
1
vote
3answers
4k views

Delete lines that contain X in specific column .csv [duplicate]

I currently use the following to delete all emails that contain EXAMPLE sed -i '/EXAMPLE/d' newname.csv However that deletes lines that have EXAMPLE in any of the columns. I want it to only delete ...
1
vote
2answers
104 views

Using join with space in key

I'm having issues merging two files using the join command: The first file is a csv file: NAIN GENIEU 01/01/1900,A,B NAIN GENIEUR 01/01/1917,C,D NAINGENIEUR 21/01/1917,E,F The second file contains ...
0
votes
1answer
2k views

Create directory and files from csv file

I have a CSV file which contains the data in the following format web_url.csv drive,http://www.drive.google.com/ wikipedia,https://en.wikipedia.org/wiki/%s wolframalpha,http://www.wolframalpha.com/...
0
votes
1answer
393 views

Splitting a file using perl

I have a csv file and like to split the file into smaller files based on column matching in the file using perl. I am working on Linux Rhel6. example: fruit1, fruit2,pricerate,quantity orange, ...
0
votes
1answer
107 views

Delete customers domain extension only (in .csv file)

I have a list of my customers emails and want to remove some that end in .br for example. I would normally do the following command: sed -i '/.br/d' customers.csv But that would also delete a ...
0
votes
2answers
166 views

.csv Keep only certain email addresses?

I have a large .csv file with my customers Name:Email I have another large list of emails that are part of this list and only want to keep those. How can I remove all contacts from the .csv file that ...
1
vote
4answers
866 views

In bash I have to ask for each row from first csv file, second csv file and the answers put to third csv file [closed]

In bash I have to ask for each row from first csv file, second csv file and the answers put to third csv file ... first file. e.g. aaa.csv: aaa bbb aaa aaa ccc fff aaa second file, e.g. bbb.csv: ...
1
vote
1answer
149 views

How to filter a basic csv by columns

How do I filter by third column of the following file list.txt [[ list of people ]] billy baxter - @baxter - b.baxter@woods.com james woods - @woods - j.woods@someother.com I currently have cat ...