Questions tagged [csv]
Files containing data arrange in a table, often with commas (hence Comma Separated Values), to separate columns. Rows are separated by newlines (but not all newlines are row separators as fields can be quoted to contain the separator newlines. Use this tag for full-fledged CSV data not the simpler ...
0
votes
0answers
13 views
Append field to csv based on conditional search from another file
Trying to append a field at the end of a large CSV file with server records:
30522,sysserver0012-softlayer.com,RDDP_Place-Live_Migration_Service-LT-
TOPSINTP-787,x86_64,8,8,VMware Virtual Platform,...
1
vote
2answers
64 views
convert tab delimited data to , delimited records with double quoted fields “ ” [closed]
I want to convert tab delimited csv into " ", add empty space if value is empty
FirstName LastName Address1 Address2 City State ZIP
John1 ...
1
vote
1answer
18 views
Awk csv treatment output malformed line to stderr
I have a script that takes several CSV files from a folder and puts them together in a gzip file with one field extracted from the file name :
awk -F";" 'BEGIN { OFS = FS }
FNR == 1 { match(FILENAME,...
0
votes
0answers
31 views
Parsing CSV that has multiple line values with carriage returns
I have a CSV that I'm trying to remove the double quote values from using the following sed one-liner. Details about this can be found in my original post to figure that out
sed -i.bak 's/^"//g;s/","/...
0
votes
1answer
26 views
awk or bash to read IDs from CSV, match rows in second CSV, change value in 2nd column
I have two .csv
items-to-change.txt is a single column list of unique ids:
218294
222084
197931
198161
197432
201021
214009
items-by-location.txt has unique ids and location number (tab delimited)
...
0
votes
2answers
63 views
SED not removing last double quote from every line
I have a CSV that was exported with double quotes around every header and value and I need them gone but without removing double quotes that might actually exist as values. For example:
"HEADER1","...
4
votes
3answers
156 views
Add lines to files to make them equal length
I have a bunch of .csv files with N columns and different number of rows (lines). I would like to add as many empty lines ;...; (N semicolons) to make them the same length. I can get the length of the ...
1
vote
1answer
11 views
How to extract data from csv file with modification to data
For eg i have a file called test.csv containing
emp1,dep1
emp2,dep2
emp3,dep1
emp4,dep3
emp5,dep2
expected output is
emp1 works in dep1
emp2 works in dep2
.
.
emp5 works in dep2
-1
votes
1answer
27 views
Truncate long line and append indicating symbol [closed]
I have sample:
"this is a long title must in a doule quotes" this_is_a_solid_long_line_without_space_or_tabs
"this is a long title must in a doule quotes" is a long line in double quotes that ...
2
votes
1answer
23 views
Layout tab/spaces [closed]
I've read this thread https://unix.stackexchange.com/a/7718/256195, that only if var doesn't contain any tab/spaces but in my case it does contain spaces, like below example:
"this is a test" ...
0
votes
3answers
40 views
awk - Print all values with a suffix
I want to use awk to print all values in a delimited string with a static suffix appended to each one.
Input:
stack,over,flow
Delimiter:
Comma
Output:
stack suffix, over suffix, flow suffix
1
vote
2answers
29 views
Add a pattern before the delimiter
I'm looking for a command to add a pattern before or the delimiter.
My sting:
1 |Chris|ubuntu
Here my delimiter is| and I want to generate a string with this input.
Output
1 = ID and Chris = ...
0
votes
0answers
32 views
Gnumeric's ssconvert corrupting file when converting .csv to .xlsx
I am using a simple bash script to search for all .csv files in a directory with a certain name and convert them to .xlsx. After that, the script removes the .csv files.
I am using ssconvert for the ...
2
votes
1answer
53 views
Extract the number of pages of PDFs recursively and dump them to a CSV
Apologies for the fairly trivial question but I am a Linux beginner.
I need to get a list of PDFs in various sub-folders of a directory structure and the number of pages of each PDF file. All saved ...
0
votes
1answer
28 views
Is the an elegant way to select rows in a file where a given column has a given value?
I have a file with a lot of lines like this
0 file:/home/knappen/somefilename.txt 7 0.2838356973995272 19 0.21823286052009455 18 0.10121158392434988 15 0....
0
votes
2answers
51 views
Organizing data by a header
I have two csv files.
File 1 contains header.
File 2 contains data
file 1 format : file1.csv
id,abc,xyz,aaa,bbb,ccc
file 2 format : file2.csv
id,source,value
1,abc,100
1,xyz,...
3
votes
6answers
621 views
Replace the third comma on each line with newline
I have these long IDs which consist of four sections:
AKJHGFGUIKL,OIUYT,KJHBTYUI,98765434567
RTYUIKHGFGH,TYUJI,TGHYJKJKLJKL,6789876
ETRYTUUI,YTYUIL,UIOKJHGFGH,34567898766
I want to put the numbers ...
0
votes
2answers
70 views
How to sort the data from CSV file by comparing data from text file using awk output it as a TXT file?
There are two files I am going to extract the data:
file1.txt:
Type Serial ID Element Hit_Possibility
Yasuo 19-2 19623 Hasaki
file2.csv:
Date,Name,Order,Hit Possibility
12-Aug,Ken,1,256
12-Aug,...
-1
votes
3answers
143 views
Field separator part of a column - incorrect parsing unix
I want to do a check the number of columns in a CSV file before processing it. The problem is that the delimiter (comma) also occurs in the text of some fields and because of that I cannot parse it ...
0
votes
2answers
97 views
How to remove duplicate items within the same line from a csv file?
I have a csv file with ~4000 lines, each one containing between 2 and 30 names separated by commas. The names are including titles (for example mr. X Adams or ms. Y Sanders). Some names exist multiple ...
1
vote
3answers
71 views
CSV file: Group and sum values based on first pattern of each value
I am using bash to create a script to find, group and sum fields in a CSV file. Each row will have comma-separated fields with each field following a similar convention. For each comma-separated field,...
3
votes
4answers
582 views
Bash: join data from two csv files
I have two csv files that contain various user data; they share one common field (username).
file A:
username ; Fullname ; mail
Bob ; Bob Hope ; bob@hope.org
file B:
username ; LastLogonTime ; ...
0
votes
1answer
73 views
Append lines from csv file to another based on column 1 matches
I have 2 .csv files, file1.csv and file2.csv. Say a line from file2.csv has the same 1st field as a line from file1.csv. I would like the full line from file1.csv to replace the full line from file2....
0
votes
3answers
85 views
AWK csv manipulation
I have a csv file containing something like this:
abc;!!!!;22
abc;!!!!;23
23;!!!!!;22
Now I want to delte every line that contains a 23 in the third column.
Since there is a possibility that there ...
0
votes
2answers
73 views
Rename a csv tab from UNIX bash script
I have CSV file that is generated from a bash script: UNIX_REPORT.sh. When you open CSV the file on the windGrr side with excel, the tab (or sheet) in the lower left corner is named UNIX_REPORT.sh.
...
0
votes
1answer
52 views
Interpolate Columnar Data
I am trying to find a way to interpolate between two lines of data in a CSV file, likely using awk. Right now, each line represents a data point at Hour 0 and Hour 6. I am looking to fill in missing ...
1
vote
3answers
73 views
Using AWK to add a new column with values to a csv file without creating empty new lines in between rows
I would like to add a new column to the end of a csv file and populating the column with values. I have used the following codes and it ends up showing new blank lines in between each record. Please ...
0
votes
2answers
76 views
Split datetime column in csv file into MM/YYYY and time column separately and write to the same csv
I am trying to use awk, split and then print but running into error
Input:
id day userId itemId
1 12/1/17 8:32 2232 tv
2 1/12/18 10:18 3232 fdfs
3 2/9/18 10:50 232 fdsf
...
-1
votes
3answers
123 views
remove the lines in CSV that contain specific pattern on the second field
How to remove the lines in .csv file that second field contain the word content?
example :
ams-hbase-log4j,content:\n#LicensedtotheApacheSoftwareFoundation(ASF)underone\n#...
1
vote
2answers
35 views
How to merge different columns in the same file?
I have three files, as shown below:
FILE 1:
DATE PGTO_CRED
20180801 50.00
20180802 150.00
20180803 130.00
20180804 110.00
20180805 200.00
FILE 2:
DATE PGTO_TOTAL
20180801 ...
1
vote
1answer
34 views
Save each line of a .csv file as a separate .csv [duplicate]
I have a .csv file with no headers and want to have each line saved as a separate .csv file with the new file name to be the value in column 2.
0
votes
0answers
42 views
file command gives incorrect encoding type
I have a csv file in hand. When I ran 'file -i filename', it shows that it's encoded as us-ascii. But when I ran cat filename | csvcut -t -e us-ascii, I got an error:
"Your file is not "us-ascii" ...
-2
votes
1answer
42 views
Store the content of a txt file into a variable with comma separated [closed]
Hi I have a file txt file with keys
a.txt:
1234
4567
4568
4564
6754
warning message
invoke from function
I need to write the content of this file into a variable parm as
1234,4567,3456,2345
I ...
2
votes
1answer
71 views
Populate a CSV file from data file with nested loops in bash
I have a file, 4.txt that contains full paths to *.cfg files as well as additional data I need to strip for the final report (5.csv).
For example /source/EDDG/env1/dom1/proj/config/test.cfg
<...
1
vote
3answers
60 views
bash: assign variable and print to stdout in same command [closed]
I'm working on a bash script that reads a csv file with fields and field values and creates a json string out of the input.
For simplicity I have a bash script that takes in two arguments simulating ...
-1
votes
3answers
117 views
remove n columns from CSV file [closed]
I have a csv file with data as below :
12,1234,28-07-2018 05:28:12.21,ABC60,3456,28-07-2018 05:28:12.21,AB60,7580,28-07-2018 06:28:12.21,PQ
I need to remove all date columns from the file through ...
1
vote
1answer
44 views
Grep regular expression to show only serial numbers (defined length containing Alpha & Digit) in CSV File
I have a lot of (huge and chaotic) CSV files with a bunch of (Cisco) serial numbers.
My goal is to extract them (BTW: And to call the Cisco API later on for getting service/support coverage feedback)...
2
votes
4answers
65 views
Move files into separate directories based on grouping defined in a CSV file
RELATED: Move all files with matching prefixes to folder based on a csv list
I have a two-column CSV file (comma-separated values), with headers:
"id","group"
"F1256","old"
"E51651","new"
"X56369","...
-2
votes
3answers
97 views
Change all but first ',' to “<COMMA>” for each row in file (bash) [duplicate]
I am using bash and have a csv file (dat.csv) that needs to be only two columns (App, Blurb) of data but due to many ',' on each row it is becoming MANY columns.
EXAMPLE OF PROBLEM csv.dat:
App , ...
4
votes
3answers
278 views
How to print the count of pattern at each line?
I need to print the count of a matching string at the end of each line.
An example for matching foo:
foo,bar,foo,foo
bar,foo,bar,bar
foo,foo,bar,bar
Result :
foo,bar,foo,foo,3
bar,foo,bar,bar,1
...
-3
votes
1answer
43 views
Script for changing CSV into Key Value (KV) format
I have a CSV file with data as below:
row_identifier,DBNAME,tblsps_name,Cur_size,Max_size,Used,Free,Percentage
tablespace,MRETF,RERETOSB15_DATA,51200,45600,14284,31316,31
tablespace,MRETF,...
0
votes
4answers
81 views
How to compare lines in a single file line by line and append the output to the same file
I have a file abc.csv that has the below contents.
a6caa290d03ae75dd5f3bf9214f0b523;CO.jar,a6caa290d03ae75dd5f3bf9214f0b523;CO.jar,
a1a5bf1547f50feea4120290343f89e4;Common.jar,...
0
votes
0answers
77 views
Reading CSV and writing CSV with Base64 encoding using cat
So I been trying to make image encoding script what I got working with 3 columns but not with 4. Example data is just with 4 rows finished data will be 100-1000 rows... Delimiter on csv is set to | ...
1
vote
1answer
142 views
Switch columns in CSV using awk? [duplicate]
I have the exact same question as this one here. Only difference is I have 3 columns only and I am trying to move the 1st column to the end.
The original file looks like this:
col1,col2,col3
2,2015-...
5
votes
3answers
375 views
sed/awk to remove double quotes " that are inside of curly braces {}
I have a csv file with data similar to this:
"1b579a5e-9701-40eb-bd36-2bc65169da99","week14_Friday-019","6907eaad-1aff-4d26-9088-ba20374b67c0","2181-019","f20af5bb-c716-42e0-9b9d-cbolf5bfecea","15-...
-1
votes
3answers
267 views
Bash to read all columns and display blank cells in each row from CSV file
I need the script to go through each column and check for blanks and group it up for me into 1 column... so e.g.
1st row: banana , 2nd row: rachel, 5th row: rachel, monica, chandler, joey, janice, ...
0
votes
1answer
28 views
How to do sum calculations in libreoffice calc, single column
I have a .csv file which has two columns and about 50 rows of data in them. One of the columns has some numbers in it. What should have worked is -
=sum<B1:B50>
the result should be displayed ...
1
vote
2answers
647 views
Using bash to swap first and second columns in CSV
I'm using bash. I have a CSV file with two columns of data that looks roughly like this
num_logins,day
253,2016-07-01
127,2016-07-02
I want to swap the first and second columns (making the date ...
2
votes
1answer
51 views
Merge multiple lines column by column based on column 1 - same file?
I'm trying to organise a CSV file with 30 columns and 1000 rows.
I want to concatenate the rows column by column if the 1st column is the same, using a vertical line (|).
Input:
3,XX,YY,123,1234,MM,-...
4
votes
3answers
240 views
How to merge first two lines of a csv column-by-column?
I have an excel file that I converted to csv. When converted, it looks like the following example (Please note that there are 100+ columns in the csv. This is a minified version):
,Product," ",Citty,...
