Questions tagged [csv]
Files containing data arrange in a table, often with commas (hence Comma Separated Values), to separate columns. Rows are separated by newlines (but not all newlines are row separators as fields can be quoted to contain the separator newlines. Use this tag for full-fledged CSV data not the simpler case of one record per line or completely unquoted (use csv-simple for that kind of data).
714
questions
1
vote
2answers
53 views
Bash script/awk to input and output a CSV file
I'm having some trouble creating a bash/awk/sed script that would take a comma-separated CSV file of three columns (firstname, lastname, date of birth), and outputs another CSV file that has the same ...
-8
votes
0answers
47 views
Convert text file to csv [closed]
SRI1GA_L2T-ACD-TEST-MC-ACDE-DEV3-MC.092503.250409Completed25-SEP-2020 04:09:5425-SEP-2020 08:28:5825-SEP-2020 08:28:58
SRII2GA_L2T-ABC-MC-ABC-TEST-MC.092303.231237Running23-SEP-2020 12:37:4325-SEP-...
5
votes
5answers
334 views
How to sort a first file (csv) based on the second file keys
I am trying to find a solution where I can sort my first file using the first column based on the second file keys
First file example (file1.csv)
COLUMN1 COlUMN2
apple fruit
dog animal
cat animal
cow ...
1
vote
1answer
54 views
How can i put a variable in an awk command
with my code im trying to sum up the values with the specific name of a column in a csv file, depending on the input of the name.
Here's my code:
#!/bin/bash
updatedata() {
index=0
while IFS=...
0
votes
2answers
48 views
Removing block of text from csv file in Linux
I have a CSV file with lines similar to:
PerformanceData|712899|713|12|5.53|VERIFIED_TRADE N CoverDistribution SecurityTransfer (BondUKTI / //-MO/W////.%)|5708|124.9
In column 6, I have a block ...
1
vote
3answers
50 views
How to use a file of search patterns to search for exact words in the final column of a csv?
This issue is covered as separate topics, but I can't get everything to work together.
I have a tsv file, here is an example line: (file1)
NODE_1_length_45927_cov_22.029055_g0_i0 WP_055195622.1 93.1 ...
2
votes
1answer
16 views
How to introduce a new header to a CSV file based on condition using gawk
I have a csv file from I wish to extract only column number 7 and 11. Depending on the value of column 7 which has either value OK or KO I wish to insert a column next to it called value where the ...
2
votes
0answers
51 views
Libre Office - convert calc/exel schedule to csv from command line
I'm truing to batch convert some exel schedules to csv format from the command-line with the following command:
libreoffice --convert-to csv *.xlsx
I'm getting:
error
xsltParseStylesheetFile : cannot ...
1
vote
1answer
18 views
Get a list of file name, and compare to a csv file while adding 2nd and 3rd column of the same 1st column
I have a csv file that looks like this:
user1,400,300
user2,250,250
user3,400,300
user1,400,300
user4,400,300
user2,250,250
and a directory that has a empty file named after the user
$ ls /dir/*
/dir/...
1
vote
2answers
35 views
Extract information between double quotes on .csv
I'm having some problems with a script parsing a .csv file
Information on .csv:
#1,13/8/2020,somedude@hotmail.com,otherdude@hotmail.com,,Subject,"Dear Dude,
Information have been updated. Please ...
0
votes
1answer
146 views
Using CSV data with awk
I am trying to find a way to take the individual data from a CSV file, and use it as a variable within a grep or awk command. Either seems appropriate but I'm not sure how to tell it to do this ...
0
votes
0answers
58 views
Split csv file on multiple csv files based on several columns headers
I have a large csv file that i tried to split into several csv files based on certain columns in the header. The objective is to generate each file in relation to the columns of the country.
For ...
0
votes
4answers
82 views
Filling in blank spaces in the first column of a csv file using Bash / awk
I am looking into automating some processes / calculations but I may first need to format a slightly awkward CSV file set. (For this I am using bash, as requested).
The csv file set follows (roughly) ...
0
votes
2answers
78 views
Remove duplicated first fields of a CSV file
I try to remove repetitions of the same value of the first column from a CSV file without changing the other cell contents and alignment (in other columns).
My txt:
ACCIDENT EP 4 STEM PERCUS,, ...
5
votes
1answer
252 views
Adding suffix to filename during for loop in bash
This has probably been asked but the existing answers don't make sense to me. I am running a loop over multiple CSV files in a folder. This folder has non-csv files also and hence I must specify the ...
0
votes
5answers
96 views
how to check validation of csv file with two fields
I have simple csv file that should contain only two non-empty fields as the following
This is example of right csv file
$ more file.csv
why_we_need_help,log_low=53687091200
whats_is_going_on,...
-1
votes
1answer
75 views
i need to generate ansible-playbook using jinja2 with csv file in ansible
i need to create jinja templates to generate ansible playbook for that i need to read datas from csv file
csv file is similar to below(file name ansi.csv)
aaa,bbb,ccc,ddd
aa01,ansi,directory,yes
aa02,...
3
votes
4answers
310 views
How to add different string to duplicates in a single column of a tab delimited file in order to make them unique
I have a 4 column tab delimited file and the last column sometimes has duplicates. This is an excerpt from that file:
chr7 116038644 116039744 GeneA
chr7 116030947 116032047 ...
0
votes
1answer
37 views
Transpose few columns from a csv file based on several columns using awk
In my previous tag, i would keeping csv columns using a awk generic code. The answer was given here by @steeldriver.
In the following, I want to transpose some columns from a csv file to get a new ...
1
vote
1answer
35 views
create file name based on CSV column data using gawk
I have a very messed up CSV file where the header looks the following:
maindata.csv
time;area;measurement1;measurement2;measurement3;.....measurement12
I was able to split the single large file based ...
0
votes
1answer
56 views
Keeping columns of csv file based on condition of columns of header line
I wish to keep only certain columns of a CSV file, based on the structure of the header line.
Description of data:
In the header line (fields of country), several fields are empty.
The number of ...
1
vote
1answer
94 views
Merge two CSV files into one using AWK
I am trying to merge two csv files using AWK. If the 1st column in file 1 matches the 1st column in file 2 I want to change file1's 3rd column to the value of file2's 2nd column. If there is no match, ...
0
votes
1answer
52 views
Get process running list in CSV
Below there sample of process running list present. From that list How can we get list process running in a csv file?
root 94701 8609 0 Jul02 ? 00:00:01 containerd-shim -namespace ...
0
votes
1answer
52 views
How to convert all text files in directory to one csv
I want to convert all text files in a directory to one csv file. I want the input to the csv to be the text in the text file labeled by the author of the text file found in the filename. For reference,...
0
votes
2answers
36 views
Showing both date and time in a single column
I am extracting time series data from NetCDF using Climate data operator.
It has the option to extract Date as well as time separately.
I have used
cdo -outputtab,date,time,value -remapnn,lon=-85....
-1
votes
8answers
175 views
Querying csv files like sql
This is apparently a popular interview question:
There are 2 CSV files with dinosaur data. We need to query them to return dinosaurs satisfying a certain condition.
There are 2 choices - use only Unix ...
-3
votes
2answers
50 views
How to find null value in file using Unix cmd
Id, name, currency
101,aaa, 10000
102,BBB,
103,ccc, 20000
I know I have an empty cell currency column. How do I find it using a terminal command?
-5
votes
1answer
68 views
vlookup using UNIX shell
I have this CSV file
ID,col1,col2,col3
1,,,FALSE
2,,,FALSE
3,,,FALSE
4,,,FALSE
and I want to join it using ID column with this other CSV file
ID,col1
2,AA
3,BB
to obtain (the real output must be ...
0
votes
1answer
44 views
How to match two csv files columns to obtain a new file
I have two csv files:
first file:
"ACCOUNT_CODE","FK_CLIENT_CODE","ENVIRONMENT","HHID"
"13445319","V8571485","SAT","IT00000000000005676070"
"10580347","V6559553","SAT","IT00000000000003952833" ...
3
votes
7answers
148 views
(AWK) Csv separating same variables in columns into different files
I have a test1.csv file with two columns
group,email
group1@email.com,user1@email.com
group1@email.com,user2@email.com
group1@email.com,user3@email.com
group1@email.com,user4@email.com
group2@email....
6
votes
3answers
559 views
Executable common files (*.pdf, etc.)
I have a number of files showing up green when I run ls. I understand these are executables, and I understand that one can make a file executable with chmod. But they are .csv and .pdf files. I don't ...
0
votes
0answers
23 views
Filtering 20GB of .csv files with grep: lookbehind doesn't seem to behave as expected [duplicate]
As per the title, I need to filter lines from about a dozen csv files totaling about 20GB.
The file contents are formatted as follows:
IdSensore,Data,Valore,Stato,idOperatore
2001,01/01/2019 00:00:...
0
votes
3answers
47 views
not able to find a file name in the directory using file name as a variable from a temporary file
I have sample CSV file contains file names
f1
f2
f3
I want to search these file name in a directory, below is my code
while read f1
do
ls -t | grep "$f1"
if[ $? = 0 ]
then
echo file ...
1
vote
1answer
52 views
Filtering CSV data into multiple separate files
I have a CSV file named file1.csv:
something;AD;sss;Andorra;nothing;type_1;sss
something222;AD;sss222;Andorra;nothing222;type_2;aaa
thing;NL;thing3;Netherlands;thing;type_2;bb
etc;US;etc;United ...
0
votes
2answers
60 views
Remove URI prefix (http:// and https://) by using awk command or shell script
I have the below data (Actual output)
http://localhost:5058/uaa/token,80
https://t-mobile.com,443
http://USERSECURITYTOKEN/payments/security/jwttoken,80
https://core.op.api.internal.t-mobile.com/v1/...
0
votes
2answers
54 views
Unable to separate valid and invalid file in unix script topic
I have three files named as a.csv, b.csv, and c.csv.
a.csv has content
1234567,11111111111111111111111111111111111111111111111111
b.csv has content like
...
0
votes
1answer
29 views
Linux remove a occurrence of sentence from a file and add add start and end
I have a script that curl a command to sms gateway to send sms and get xml as response. Each response are appended to a single file for future xsltproc process for creating csv.
A single response is ...
2
votes
3answers
161 views
Add a column to file in linux at beginning of line if length is less than 4
I have a file like below on linux folder. There should be 4 columns in file with comma separated values.But in some lines there are only 3 columns. I want to add a blank at the beginning of line when ...
1
vote
1answer
17 views
Transform only certain entries in comma-separate file
I have a netlist file that contains some bitblasted signals. I want to change the signal name like below.
Its like
\abc[0],\abc[1].....
\xyz[0],\xyz[2]....
And I want it to be like
abc_0,abc_1....
...
0
votes
2answers
31 views
Replacing parameters in loop with values from a csv
I am trying to write a bash for loop that execute a python script multiple times, such that it's like this
for i from 1 to 10 do
arg1 = 'path/to/directory1'
arg2 = 'path/to/directory2'
...
1
vote
2answers
50 views
I want to only remove grouped commas from this csv to change the number of variables to two
I have a csv where the first few rows look like this
c("4288", "57534"),MIB1
c("2272", "2385"),FHIT
c("5550", "10531", "56239"),PREP
c("25809", "23669"),TTLL1
I want to manipulate the number of ...
0
votes
2answers
31 views
Find a value in a specific position when the only information you have is the position
I have a csv file approximately 16,000 rows long, with two fields. The first field contains a list of numeric values, and the second field contains a list of first and names delimited by semi-colons ...
-1
votes
4answers
49 views
How to print columnName=Value in new row by reading CSV file?
I have a CSV file Foo.csv with the following data:
Name,Age,Education,Address
Alex,22,Computer Science, TX
Ron,24,Computer Science, AU
I am expecting the following output:
Name=Alex
Age=22
...
0
votes
3answers
93 views
Extract data from the nested parenthesis in bash
I'd need a way to extract certain data and store it in file for auditing and migration.
Here's my data look like.
network vlan internal-vlan {
description "internal-use"
interfaces {
...
0
votes
4answers
77 views
Compare two files, if the columns are same then replace next column with some value
How do I replace a column value in a file when compared with another file?
I have two files test1.csv and test2.csv; I need to replace the empdep column in test1.csv if it has the value as "sign*". ...
-1
votes
2answers
71 views
How I can extract just some fields from a CSV line of text
I need help to find out a way to extract specific information of the lines below using Linux commands.
391,(INSIDE-A),to,(OUTSIDE-A),source,static,SRV_I_N1909,SRV_NAT_I_N1909,destination,static,...
1
vote
1answer
54 views
sort datetimestamp column in a csv file from the command line
I have a csv file contains a datetime column format as
1 Mon Mar 02 07:42:29 UT 2020
2 Tue Mar 03 09:02:09 UT 2020
3 Mon Mar 02 01:21:29 UT 2020
I am trying to use the sort, but I dont think ...
0
votes
3answers
57 views
Removing everything in a column after a specific embeded character in a csv file
I need to remove all the contents that appear in a column after a specific character in CSV file.
This is the format it is currently in:
2020-01-01T12:00:00,Transaction Description,Initiator,$00.00
...
0
votes
2answers
127 views
Remove interval double quotes in a CSV separated by comma and encapsulated by double quotes
Perhaps I'm out of luck, because my double quoted comma separated CSV file has double quotes and commas within useful text.
So I want to turn this:
"record 1","name 1","text 1, text 2"
"record 2","...
1
vote
2answers
56 views
Remove duplicates csv based on first value keeping the longest line between duplicates
I have many different csv files in a folder (megadrive.txt, snes.txt) like this:
Aerial Assault (USA);Aerial Assault (USA);Sega Master System;;;;;;;;;0;;;;;
Aerial Assault (USA);Aerial Assault (USA);...
