Questions tagged [csv-simple]

Text files containing simple comma separated values. Use this for data that is a subset of real CSV without newlines in (quoted) field content, that can be handled with line oriented tools without the need for a real parser.

Filter by
Sorted by
Tagged with
0 votes
0 answers
43 views

Grouping and aggregating data

Can you please guide me, on how to group and aggregate the below value using a shell script (using awk) or guide me on which is the best option? Input textfile: 900|1000|currentaccount|usd|66.66|gbp|...
user avatar
0 votes
2 answers
65 views

Reformatting a list of elements using bash/sed

I have a CSV file with a list like this: URL,Domain,anchor https://example1.com,Example1,Category1 I need to reformat this into HTML, like this: <li><a href="https://example1.com" ...
user avatar
-1 votes
2 answers
90 views

awk/sed filter example for column manipulation based on data in other columns

I am trying to find an example of how I may effeciently use sed or awk to manipulate csv data in a specific column (col 1 for instance) if the data in col 2 AND col 5 are not equal to "null"....
user avatar
0 votes
2 answers
42 views

extract file to generate and rename differenf files

I have a csv (in which firt and second column are separeted by ",") file like this: Column1,Column2 4e,info1 4t,info2 45t,info3 3,info4 I would like to obtained 4 different file, one for ...
user avatar
  • 161
0 votes
5 answers
203 views

Bash: title-case a csv field

I have this input file on a CentOS system: 1,,,,ivan petrov,,67, 2,2,,,Vasia pupkin,director,8, 3,,,,john Lenon,,, The task is to change it to: 1,,,,Ivan Petrov,,67, 2,2,,,Vasia Pupkin,director,8, 3,,...
user avatar
0 votes
4 answers
68 views

format a CSV file properly so as to fetch data from the CSV correctly

I have a CSV file as below : I want to remove the "INITIAL OFFER" block from this file and remain with only "FINAL OFFER" block I also want to remove comma(,) from the first field ...
user avatar
2 votes
3 answers
159 views

Adding and subtracting multiple columns into new column

I have copied a partial csv file. publish_date,headline_text,likes_count,comments_count,shares_count,love_count,wow_count,haha_count,sad_count,thankful_count,angry_count 20030219,aba decides against ...
user avatar
  • 35
0 votes
3 answers
93 views

print the number of column and "ERROR" if it has an empty field else "CORRECT" if it has no empty fields

I have a csv that looks as follows: column1, column2, column3, column4, column5, 1,,,,5, 1,2,3,,, 1,2,3,4,5 1,2,3,4,5, 1,2,,,5, Using awk I want to print the number of all the rows and a comment &...
user avatar
0 votes
1 answer
82 views

awk: add two columns to csv that are the multiplication of a constant by the last column

I am trying to add two new columns to a csv file, that contain the multiplication of the last column to a constant. This last column is formed by the pound symbol and followed by a number. Input: id,...
user avatar
4 votes
3 answers
332 views

How to do calculation for each row

I had a csv file of data such like that when read into shell: name,income,reward,payment Jackson,10000,2000,1000 Paul,2500,700,200 Louis,5000,100,1800 and I want to find the net earning for each ...
user avatar
  • 43
0 votes
1 answer
356 views

How to convert a CSV with multiple sub-row items (from a column) to new columns in Awk?

I have a CSV file with the general format shown in the picture below. In that CSV there are multiple rows that belong to a certain column (desc) and I would like to to extract those items and add them ...
user avatar
  • 1,400
0 votes
3 answers
2k views

Compare two columns of a csv file and show unmatched records only

I have a csv file in below format 1311,0008,a131,a131,7039 1311,0008,a131,a132,7039 1711,2046,a626565,a626566,7099 1711,2048,a626565,a626565,7035 What I want is to compare column 3 and 4 only and if ...
user avatar
4 votes
3 answers
236 views

How to add column when the number of columns in file is 2

I am trying to write awk for a file I have. Example of the dataset is S,CV0110,1235 S,1234 D,CQ120,3245 P,7894 Desired outcome is as follows (added empty field when the number of fields in a row is 2)...
user avatar
  • 69
1 vote
2 answers
162 views

Using loop in csv file create separate file for each line

I have a csv file containing a,b,c d,e,f g,h,i now using this I want to create 3 separate files a.txt, d.txt, g.txt (named after the first field) for each record of that csv. Each file will ...
user avatar
0 votes
4 answers
120 views

XML Data to CSV

Need some quick help as I am not able to get the desired output: 2020-05-19 19:03:07,135 INFO [Container : 8504] [HttpUtil.java]requestXML: <?xml version="1.0"? ><COMMAND><TYPE>...
user avatar
0 votes
1 answer
25 views

How to use a like function to get the data having comma

We have a data like below Column1,|Column2,|Col|Abc|Column3,&TableName --- ( Here my delimiter is "&" ) Is there any way i can make an output as below Column1,Column2,Column3,&...
user avatar
  • 282
1 vote
2 answers
53 views

I want to only remove grouped commas from this csv to change the number of variables to two

I have a csv where the first few rows look like this c("4288", "57534"),MIB1 c("2272", "2385"),FHIT c("5550", "10531", "56239"),PREP c("25809", "23669"),TTLL1 I want to manipulate the number of ...
user avatar
  • 23
0 votes
1 answer
758 views

SED not removing last double quote from every line

I have a CSV that was exported with double quotes around every header and value and I need them gone but without removing double quotes that might actually exist as values. For example: "HEADER1","...
user avatar
0 votes
3 answers
2k views

How to extract lines when two columns strings are not equal

I have a CSV file that looks like this format: text1,text2,string1,string2 text3,text3,string3,string2 text4,text5,string1,string2 text6,text6,string6,string7 I want to extract rows when column1 and ...
user avatar
2 votes
5 answers
4k views

awk: split column, print left side of delimiter and paste in the middle of csv

Input file: AAA, BBB:XXX, CCC, DDD, EEE, FFF, GGG, HHH Output should be: AAA, BBB, BBB:XXX, CCC, DDD, EEE, FFF, GGG, HHH I thought about something like: awk -F, '{n=split($2,a,":"); a[n]} {$2=$NF"...
user avatar
  • 157
0 votes
2 answers
1k views

Append to second column

There is a csv file ip,time,name 1.1.1.1,2018-08-31 10:01:03,postgres 2.1.1.1,2018-08-31 12:51:13,oracle 3.1.1.1,2018-08-31 14:32:53,sql I need output as below ip,time,name 1.1.1.1,2018-08-31 10:...
user avatar
0 votes
3 answers
568 views

Replace multiple variable values in a file with given input values (from a file)

If i have two input file with contents like below: How i can replace network_name (eg:P1MSVmgmtvM replace with /rest/ethernet-networks/2bf8bc44-5b31-4e6c-99ed-d68b1e9acf9e). File1: (input file) ...
user avatar
0 votes
1 answer
49 views

print each Network set in separate same group

Currently my csv input looks like below: network_name,network_set P1MSVmgmtvM,Data_NetworkSet_A P1MSVvMotion,Data_NetworkSet_A P2MSVmgmtvM,Data_NetworkSet_B P2MSVvMotion,Data_NetworkSet_B ...
user avatar
1 vote
3 answers
632 views

remove duplicated pattern/entries within each field in CSV file

How do I remove duplicated entries within each separate fields with below sample as data. 0x,9.4,,,#0,#UNIX#unix,#cli#L#فا#0#فا#0#L#SE#Cli#SE,#فارسی#فارسی#۱#1#١#1,bsh,#V & v expected output(...
user avatar
  • 38.2k
0 votes
3 answers
2k views

Using AWK To Extract Numbers From .CSV File

I have a .CSV file that I need to extract numbers from. Example of file: #File of numbers 1,2,3,4,5 6,7,8,9,10 I want to use awk (or another utility) to extract the first number and second number ...
user avatar
  • 1
9 votes
4 answers
7k views

How to insert CSV data into an SQLite table via a shell pipe?

I have written a program that outputs the result to the standard output in strict pure CSV form (every line represents a single record and contain the same set of comma-separated fields, fields only ...
user avatar
  • 16.2k
3 votes
1 answer
129 views

Why is column adding a newline in the middle of my row where one is not present in the original data?

I'm analyzing some packetfilter logs and wanted to make a nice table of some output, which normally works fine when I use column -t. I can't use a tab as my output field separator (OFS) in this case ...
user avatar
  • 406
0 votes
2 answers
186 views

Checking if a text file has all entries and is formatted correctly

I have a file that is like this 0001;4;5;30;33;41;52 0002;9;37;39;41;43;49 0003;10;11;29;30;36;47 0004;1;5;6;27;42;59 0005;1;2;6;16;19;46 0006;7;13;19;22;40;47 0007;3;5;20;21;38;56 ... etc The first ...
user avatar
  • 4,204
0 votes
1 answer
1k views

How can I replace Nth delimiter in a csv file?

I have a comma seperated csv file as a sample like: 1,92345,92345,Dear user, this is your amount , 2016-10-10 2,92345,92345,Dear user, this is your amount , 2016-10-09 I need to replace 4th comma , ...
user avatar
6 votes
7 answers
1k views

How to remove duplicate value in a tab-delimited text file

I have a tab delimited column text like below A B1 B1 C1 B B2 D2 C C12 C13 C13 D D3 D5 D9 G F2 F2 how could I convert the above table ...
user avatar
  • 119
0 votes
2 answers
488 views

Pivoting Dynamic row data to columns wrt unique values in another column

I'm stuck with a particular requirement which requires the conversion of dynamic row data to column fields. Suppose I am having a file "output.csv" with the contents like: TimeStamp,Block,...
user avatar
  • 3
8 votes
1 answer
10k views

Print each field of CSV on newline without knowing number of fields

I was playing with IFS today and created a quick text file with a list of numbers separated by commas on 1 line. 1,2,3,4,5 I then tried to write a script to print each number on a newline. I was ...
user avatar
-1 votes
2 answers
2k views

Add a header field, and the file's name at the end of the data lines, for all the csv files in a folder [closed]

I want to add filename (without extension) at the end for all the lines for all the csv files in a folder. All the files are having the same header. Let's say I have two files a.csv and b.csv in a ...
user avatar
  • 39
2 votes
5 answers
4k views

Use AWK to split substring by last n characters in to a new column

I have a large .csv file where I need to split a specific column by string length. I'm trying to take the last 6 characters of column 2 and move them into a new column. Current: 3102017,90131112,0,...
user avatar
3 votes
3 answers
5k views

Count the number of occurrences of a column value in a TSV file with AWK

I have a TSV tab-separated file with 3 cols: ID\tTEXT\tTYPE To print the TYPE column I do cat /dataset.csv | awk -F $'\t' '{print $3}' Those values are an enumeration of values like {CLASS_A,...
user avatar
1 vote
2 answers
213 views

text processing : extract parts of a file and write them into a single line

I have several files formatted like this: ... <title> Field1 : DATA_FIELD_1</title> ... <i class="blablabla"></i> <b>Field2 : </b> <span>DATA_FIELD_2</...
user avatar
0 votes
2 answers
3k views

Calculate sum of a column using shellscript

I have a csv file as text.csv and want to calculate the total salary of all employees. The data in it is as follows: Id,name,salary,designation 12,joe,thomas,253637,IT,admin 25,miacheal,364829,...
user avatar
  • 7
0 votes
1 answer
434 views

appending three CSV files [duplicate]

I have three CSV files which have the same same row count. 1.csv 1,2,3 4,5,6 7,8,9 2.csv 4 5 6 3.csv 8 9 10 After combining I want the result as result.csv 1,2,3,4,8 4,5,6,5,9 7,8,9,6,10 So I ...
user avatar
  • 133
1 vote
3 answers
307 views

merge multiple lines of data having one common field

I have a large file with 9+ rows of data seperates with semi-colon(;) and I want to merge data in column 3 (seperating with ,) the lines where data in column 5 matches. The data is held on a Linux ...
user avatar
  • 13
1 vote
2 answers
120 views

How to convert uA/mA to uA in each lines of csv file using c shell?

I have a *.csv file as bellow. What I want to do is converting all units to one unit (uA). Anyone can help to do this in c shell??
user avatar
  • 21
2 votes
4 answers
12k views

How to check for null fields in CSV file?

My CSV file will look like this (pipe separated): apple|banana|pear||grapefruit lemon|lime|damson|jackfruit |tangerine|nectarine|plum apricot|orange|pineapple|coconut| (Assume that any sort of ...
user avatar
4 votes
4 answers
222 views

Double quote value assignments stored in a CSV?

I have a file that contains text as follows: dt=2016-06-30,path=path1,site=US,mobile=1 dt=2016-06-21,path=path2,site=UK,mobile=0 I want to convert it to text with double-quoted values in the key-...
user avatar
3 votes
1 answer
779 views

Command to "fill down" columns in text file, a la the Excel fill down function

I have a text file with rows, and columns within those rows. I want to, essentially, replicate the Excel "fill down" function. In other words, if there is a blank "cell" on a line, it will look to the ...
user avatar
  • 141
2 votes
1 answer
35 views

Update table header regarding file name

I have several files named has follow (I just give 2 names as examples)(note: they have a semi colon ; field separator): component1_component2_component3.csv e.g. almourol_gfdl_85.csv ...
user avatar
  • 548
0 votes
1 answer
46 views

Append filenames to csv-file if matching id

I have two files file1 159211;exampleimage002.JPG 159212;example.jpg 159212;example with space.jpg ... file2 13;"OVET";"O1";"OVET/ULKO-OVET";159212;"JELDWEN BASIC 0015 9*21 VASEN";"KPL";1;345;1;0;"...
user avatar
1 vote
2 answers
41 views

Data extraction problem based on upper case words

I have a data file in the following format: source.dat (Example for illustration purposes.) ALBANIA Duck 1000 BELGIUM CHARLEROI Donald Duck 10234 CYPRUS J. Mickey 567 I'm looking for a Linux script ...
user avatar
  • 4,645
0 votes
3 answers
60 views

Change CSV records sequentially

I want to replace the text sequentially in a file. The file's contents are: 1,01-JUN-2016,ABC 2,01-JUN-2016,ABC 3,01-JUN-2016,ABC 4,01-JUN-2016,ABC 5,01-JUN-2016,ABC 6,02-JUN-2016,ABC 7,02-JUN-2016,...
user avatar
  • 1
1 vote
2 answers
2k views

Convert command output to csv, with time stamp

I want to test a speed connection with terminal command: ./speedtest-cli And it returns This: Retrieving speedtest.net configuration... Retrieving speedtest.net server list... Testing from Moscow, ...
user avatar
  • 113
4 votes
1 answer
2k views

AWK, Sum of category

I have tons of CSV files with similar content. The values are usually comma separated and they look like this. product_a, domestic, 500 product_a, abroad, 15 product_b, domestic, 313 ...
user avatar
  • 43
3 votes
1 answer
666 views

Create table with frequency of unique names retrieved from multiple .csv files

I have 32 CSV files containing fetched information from a database. I need to make a frequency table in TSV/CSV format, where the names of the rows are the name of each file, and the names of the ...
user avatar
  • 377