Questions tagged [csv]

Files containing data arrange in a table, often with commas (hence Comma Separated Values), to separate columns. Rows are separated by newlines (but not all newlines are row separators as fields can be quoted to contain the separator newlines. Use this tag for full-fledged CSV data not the simpler case of one record per line or completely unquoted (use csv-simple for that kind of data).

Filter by
Sorted by
Tagged with
1
vote
2answers
53 views

Bash script/awk to input and output a CSV file

I'm having some trouble creating a bash/awk/sed script that would take a comma-separated CSV file of three columns (firstname, lastname, date of birth), and outputs another CSV file that has the same ...
-8
votes
0answers
47 views

Convert text file to csv [closed]

SRI1GA_L2T-ACD-TEST-MC-ACDE-DEV3-MC.092503.250409Completed25-SEP-2020 04:09:5425-SEP-2020 08:28:5825-SEP-2020 08:28:58 SRII2GA_L2T-ABC-MC-ABC-TEST-MC.092303.231237Running23-SEP-2020 12:37:4325-SEP-...
5
votes
5answers
334 views

How to sort a first file (csv) based on the second file keys

I am trying to find a solution where I can sort my first file using the first column based on the second file keys First file example (file1.csv) COLUMN1 COlUMN2 apple fruit dog animal cat animal cow ...
1
vote
1answer
54 views

How can i put a variable in an awk command

with my code im trying to sum up the values with the specific name of a column in a csv file, depending on the input of the name. Here's my code: #!/bin/bash updatedata() { index=0 while IFS=...
0
votes
2answers
48 views

Removing block of text from csv file in Linux

I have a CSV file with lines similar to: PerformanceData|712899|713|12|5.53|VERIFIED_TRADE N CoverDistribution SecurityTransfer (BondUKTI / //-MO/W////.%)|5708|124.9 In column 6, I have a block ...
1
vote
3answers
50 views

How to use a file of search patterns to search for exact words in the final column of a csv?

This issue is covered as separate topics, but I can't get everything to work together. I have a tsv file, here is an example line: (file1) NODE_1_length_45927_cov_22.029055_g0_i0 WP_055195622.1 93.1 ...
2
votes
1answer
16 views

How to introduce a new header to a CSV file based on condition using gawk

I have a csv file from I wish to extract only column number 7 and 11. Depending on the value of column 7 which has either value OK or KO I wish to insert a column next to it called value where the ...
2
votes
0answers
51 views

Libre Office - convert calc/exel schedule to csv from command line

I'm truing to batch convert some exel schedules to csv format from the command-line with the following command: libreoffice --convert-to csv *.xlsx I'm getting: error xsltParseStylesheetFile : cannot ...
1
vote
1answer
18 views

Get a list of file name, and compare to a csv file while adding 2nd and 3rd column of the same 1st column

I have a csv file that looks like this: user1,400,300 user2,250,250 user3,400,300 user1,400,300 user4,400,300 user2,250,250 and a directory that has a empty file named after the user $ ls /dir/* /dir/...
1
vote
2answers
35 views

Extract information between double quotes on .csv

I'm having some problems with a script parsing a .csv file Information on .csv: #1,13/8/2020,somedude@hotmail.com,otherdude@hotmail.com,,Subject,"Dear Dude, Information have been updated. Please ...
0
votes
1answer
146 views

Using CSV data with awk

I am trying to find a way to take the individual data from a CSV file, and use it as a variable within a grep or awk command. Either seems appropriate but I'm not sure how to tell it to do this ...
0
votes
0answers
58 views

Split csv file on multiple csv files based on several columns headers

I have a large csv file that i tried to split into several csv files based on certain columns in the header. The objective is to generate each file in relation to the columns of the country. For ...
0
votes
4answers
82 views

Filling in blank spaces in the first column of a csv file using Bash / awk

I am looking into automating some processes / calculations but I may first need to format a slightly awkward CSV file set. (For this I am using bash, as requested). The csv file set follows (roughly) ...
0
votes
2answers
78 views

Remove duplicated first fields of a CSV file

I try to remove repetitions of the same value of the first column from a CSV file without changing the other cell contents and alignment (in other columns). My txt: ACCIDENT EP 4 STEM PERCUS,, ...
5
votes
1answer
252 views

Adding suffix to filename during for loop in bash

This has probably been asked but the existing answers don't make sense to me. I am running a loop over multiple CSV files in a folder. This folder has non-csv files also and hence I must specify the ...
0
votes
5answers
96 views

how to check validation of csv file with two fields

I have simple csv file that should contain only two non-empty fields as the following This is example of right csv file $ more file.csv why_we_need_help,log_low=53687091200 whats_is_going_on,...
-1
votes
1answer
75 views

i need to generate ansible-playbook using jinja2 with csv file in ansible

i need to create jinja templates to generate ansible playbook for that i need to read datas from csv file csv file is similar to below(file name ansi.csv) aaa,bbb,ccc,ddd aa01,ansi,directory,yes aa02,...
3
votes
4answers
310 views

How to add different string to duplicates in a single column of a tab delimited file in order to make them unique

I have a 4 column tab delimited file and the last column sometimes has duplicates. This is an excerpt from that file: chr7 116038644 116039744 GeneA chr7 116030947 116032047 ...
0
votes
1answer
37 views

Transpose few columns from a csv file based on several columns using awk

In my previous tag, i would keeping csv columns using a awk generic code. The answer was given here by @steeldriver. In the following, I want to transpose some columns from a csv file to get a new ...
1
vote
1answer
35 views

create file name based on CSV column data using gawk

I have a very messed up CSV file where the header looks the following: maindata.csv time;area;measurement1;measurement2;measurement3;.....measurement12 I was able to split the single large file based ...
0
votes
1answer
56 views

Keeping columns of csv file based on condition of columns of header line

I wish to keep only certain columns of a CSV file, based on the structure of the header line. Description of data: In the header line (fields of country), several fields are empty.  The number of ...
1
vote
1answer
94 views

Merge two CSV files into one using AWK

I am trying to merge two csv files using AWK. If the 1st column in file 1 matches the 1st column in file 2 I want to change file1's 3rd column to the value of file2's 2nd column. If there is no match, ...
0
votes
1answer
52 views

Get process running list in CSV

Below there sample of process running list present. From that list How can we get list process running in a csv file? root 94701 8609 0 Jul02 ? 00:00:01 containerd-shim -namespace ...
0
votes
1answer
52 views

How to convert all text files in directory to one csv

I want to convert all text files in a directory to one csv file. I want the input to the csv to be the text in the text file labeled by the author of the text file found in the filename. For reference,...
0
votes
2answers
36 views

Showing both date and time in a single column

I am extracting time series data from NetCDF using Climate data operator. It has the option to extract Date as well as time separately. I have used cdo -outputtab,date,time,value -remapnn,lon=-85....
-1
votes
8answers
175 views

Querying csv files like sql

This is apparently a popular interview question: There are 2 CSV files with dinosaur data. We need to query them to return dinosaurs satisfying a certain condition. There are 2 choices - use only Unix ...
-3
votes
2answers
50 views

How to find null value in file using Unix cmd

Id, name, currency 101,aaa, 10000 102,BBB, 103,ccc, 20000 I know I have an empty cell currency column. How do I find it using a terminal command?
-5
votes
1answer
68 views

vlookup using UNIX shell

I have this CSV file ID,col1,col2,col3 1,,,FALSE 2,,,FALSE 3,,,FALSE 4,,,FALSE and I want to join it using ID column with this other CSV file ID,col1 2,AA 3,BB to obtain (the real output must be ...
0
votes
1answer
44 views

How to match two csv files columns to obtain a new file

I have two csv files: first file: "ACCOUNT_CODE","FK_CLIENT_CODE","ENVIRONMENT","HHID" "13445319","V8571485","SAT","IT00000000000005676070" "10580347","V6559553","SAT","IT00000000000003952833" ...
3
votes
7answers
148 views

(AWK) Csv separating same variables in columns into different files

I have a test1.csv file with two columns group,email group1@email.com,user1@email.com group1@email.com,user2@email.com group1@email.com,user3@email.com group1@email.com,user4@email.com group2@email....
6
votes
3answers
559 views

Executable common files (*.pdf, etc.)

I have a number of files showing up green when I run ls. I understand these are executables, and I understand that one can make a file executable with chmod. But they are .csv and .pdf files. I don't ...
0
votes
0answers
23 views

Filtering 20GB of .csv files with grep: lookbehind doesn't seem to behave as expected [duplicate]

As per the title, I need to filter lines from about a dozen csv files totaling about 20GB. The file contents are formatted as follows: IdSensore,Data,Valore,Stato,idOperatore 2001,01/01/2019 00:00:...
0
votes
3answers
47 views

not able to find a file name in the directory using file name as a variable from a temporary file

I have sample CSV file contains file names f1 f2 f3 I want to search these file name in a directory, below is my code while read f1 do ls -t | grep "$f1" if[ $? = 0 ] then echo file ...
1
vote
1answer
52 views

Filtering CSV data into multiple separate files

I have a CSV file named file1.csv: something;AD;sss;Andorra;nothing;type_1;sss something222;AD;sss222;Andorra;nothing222;type_2;aaa thing;NL;thing3;Netherlands;thing;type_2;bb etc;US;etc;United ...
0
votes
2answers
60 views

Remove URI prefix (http:// and https://) by using awk command or shell script

I have the below data (Actual output) http://localhost:5058/uaa/token,80 https://t-mobile.com,443 http://USERSECURITYTOKEN/payments/security/jwttoken,80 https://core.op.api.internal.t-mobile.com/v1/...
0
votes
2answers
54 views

Unable to separate valid and invalid file in unix script topic

I have three files named as a.csv, b.csv, and c.csv. a.csv has content 1234567,11111111111111111111111111111111111111111111111111 b.csv has content like ...
0
votes
1answer
29 views

Linux remove a occurrence of sentence from a file and add add start and end

I have a script that curl a command to sms gateway to send sms and get xml as response. Each response are appended to a single file for future xsltproc process for creating csv. A single response is ...
2
votes
3answers
161 views

Add a column to file in linux at beginning of line if length is less than 4

I have a file like below on linux folder. There should be 4 columns in file with comma separated values.But in some lines there are only 3 columns. I want to add a blank at the beginning of line when ...
1
vote
1answer
17 views

Transform only certain entries in comma-separate file

I have a netlist file that contains some bitblasted signals. I want to change the signal name like below. Its like \abc[0],\abc[1]..... \xyz[0],\xyz[2].... And I want it to be like abc_0,abc_1.... ...
0
votes
2answers
31 views

Replacing parameters in loop with values from a csv

I am trying to write a bash for loop that execute a python script multiple times, such that it's like this for i from 1 to 10 do arg1 = 'path/to/directory1' arg2 = 'path/to/directory2' ...
1
vote
2answers
50 views

I want to only remove grouped commas from this csv to change the number of variables to two

I have a csv where the first few rows look like this c("4288", "57534"),MIB1 c("2272", "2385"),FHIT c("5550", "10531", "56239"),PREP c("25809", "23669"),TTLL1 I want to manipulate the number of ...
0
votes
2answers
31 views

Find a value in a specific position when the only information you have is the position

I have a csv file approximately 16,000 rows long, with two fields. The first field contains a list of numeric values, and the second field contains a list of first and names delimited by semi-colons ...
-1
votes
4answers
49 views

How to print columnName=Value in new row by reading CSV file?

I have a CSV file Foo.csv with the following data: Name,Age,Education,Address Alex,22,Computer Science, TX Ron,24,Computer Science, AU I am expecting the following output: Name=Alex Age=22 ...
0
votes
3answers
93 views

Extract data from the nested parenthesis in bash

I'd need a way to extract certain data and store it in file for auditing and migration. Here's my data look like. network vlan internal-vlan { description "internal-use" interfaces { ...
0
votes
4answers
77 views

Compare two files, if the columns are same then replace next column with some value

How do I replace a column value in a file when compared with another file? I have two files test1.csv and test2.csv; I need to replace the empdep column in test1.csv if it has the value as "sign*". ...
-1
votes
2answers
71 views

How I can extract just some fields from a CSV line of text

I need help to find out a way to extract specific information of the lines below using Linux commands. 391,(INSIDE-A),to,(OUTSIDE-A),source,static,SRV_I_N1909,SRV_NAT_I_N1909,destination,static,...
1
vote
1answer
54 views

sort datetimestamp column in a csv file from the command line

I have a csv file contains a datetime column format as 1 Mon Mar 02 07:42:29 UT 2020 2 Tue Mar 03 09:02:09 UT 2020 3 Mon Mar 02 01:21:29 UT 2020 I am trying to use the sort, but I dont think ...
0
votes
3answers
57 views

Removing everything in a column after a specific embeded character in a csv file

I need to remove all the contents that appear in a column after a specific character in CSV file. This is the format it is currently in: 2020-01-01T12:00:00,Transaction Description,Initiator,$00.00 ...
0
votes
2answers
127 views

Remove interval double quotes in a CSV separated by comma and encapsulated by double quotes

Perhaps I'm out of luck, because my double quoted comma separated CSV file has double quotes and commas within useful text. So I want to turn this: "record 1","name 1","text 1, text 2" "record 2","...
1
vote
2answers
56 views

Remove duplicates csv based on first value keeping the longest line between duplicates

I have many different csv files in a folder (megadrive.txt, snes.txt) like this: Aerial Assault (USA);Aerial Assault (USA);Sega Master System;;;;;;;;;0;;;;; Aerial Assault (USA);Aerial Assault (USA);...

1
2 3 4 5
15