Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [csv]

Files containing data arrange in a table, often with commas (hence Comma Separated Values), to separate columns. Rows are separated by newlines (but not all newlines are row separators as fields can be quoted to contain the separator newlines. Use this tag for full-fledged CSV data not the simpler ...

0
votes
0answers
13 views

Append field to csv based on conditional search from another file

Trying to append a field at the end of a large CSV file with server records: 30522,sysserver0012-softlayer.com,RDDP_Place-Live_Migration_Service-LT- TOPSINTP-787,x86_64,8,8,VMware Virtual Platform,...
1
vote
2answers
64 views

convert tab delimited data to , delimited records with double quoted fields “ ” [closed]

I want to convert tab delimited csv into " ", add empty space if value is empty FirstName LastName Address1 Address2 City State ZIP John1 ...
1
vote
1answer
18 views

Awk csv treatment output malformed line to stderr

I have a script that takes several CSV files from a folder and puts them together in a gzip file with one field extracted from the file name : awk -F";" 'BEGIN { OFS = FS } FNR == 1 { match(FILENAME,...
0
votes
0answers
31 views

Parsing CSV that has multiple line values with carriage returns

I have a CSV that I'm trying to remove the double quote values from using the following sed one-liner. Details about this can be found in my original post to figure that out sed -i.bak 's/^"//g;s/","/...
0
votes
1answer
26 views

awk or bash to read IDs from CSV, match rows in second CSV, change value in 2nd column

I have two .csv items-to-change.txt is a single column list of unique ids: 218294 222084 197931 198161 197432 201021 214009 items-by-location.txt has unique ids and location number (tab delimited) ...
0
votes
2answers
63 views

SED not removing last double quote from every line

I have a CSV that was exported with double quotes around every header and value and I need them gone but without removing double quotes that might actually exist as values. For example: "HEADER1","...
4
votes
3answers
156 views

Add lines to files to make them equal length

I have a bunch of .csv files with N columns and different number of rows (lines). I would like to add as many empty lines ;...; (N semicolons) to make them the same length. I can get the length of the ...
1
vote
1answer
11 views

How to extract data from csv file with modification to data

For eg i have a file called test.csv containing emp1,dep1 emp2,dep2 emp3,dep1 emp4,dep3 emp5,dep2 expected output is emp1 works in dep1 emp2 works in dep2 . . emp5 works in dep2
-1
votes
1answer
27 views

Truncate long line and append indicating symbol [closed]

I have sample: "this is a long title must in a doule quotes" this_is_a_solid_long_line_without_space_or_tabs "this is a long title must in a doule quotes" is a long line in double quotes that ...
2
votes
1answer
23 views

Layout tab/spaces [closed]

I've read this thread https://unix.stackexchange.com/a/7718/256195, that only if var doesn't contain any tab/spaces but in my case it does contain spaces, like below example: "this is a test" ...
0
votes
3answers
40 views

awk - Print all values with a suffix

I want to use awk to print all values in a delimited string with a static suffix appended to each one. Input: stack,over,flow Delimiter: Comma Output: stack suffix, over suffix, flow suffix
1
vote
2answers
29 views

Add a pattern before the delimiter

I'm looking for a command to add a pattern before or the delimiter. My sting: 1 |Chris|ubuntu Here my delimiter is| and I want to generate a string with this input. Output 1 = ID and Chris = ...
0
votes
0answers
32 views

Gnumeric's ssconvert corrupting file when converting .csv to .xlsx

I am using a simple bash script to search for all .csv files in a directory with a certain name and convert them to .xlsx. After that, the script removes the .csv files. I am using ssconvert for the ...
2
votes
1answer
53 views

Extract the number of pages of PDFs recursively and dump them to a CSV

Apologies for the fairly trivial question but I am a Linux beginner. I need to get a list of PDFs in various sub-folders of a directory structure and the number of pages of each PDF file. All saved ...
0
votes
1answer
28 views

Is the an elegant way to select rows in a file where a given column has a given value?

I have a file with a lot of lines like this 0 file:/home/knappen/somefilename.txt 7 0.2838356973995272 19 0.21823286052009455 18 0.10121158392434988 15 0....
0
votes
2answers
51 views

Organizing data by a header

I have two csv files. File 1 contains header. File 2 contains data file 1 format : file1.csv id,abc,xyz,aaa,bbb,ccc file 2 format : file2.csv id,source,value 1,abc,100 1,xyz,...
3
votes
6answers
621 views

Replace the third comma on each line with newline

I have these long IDs which consist of four sections: AKJHGFGUIKL,OIUYT,KJHBTYUI,98765434567 RTYUIKHGFGH,TYUJI,TGHYJKJKLJKL,6789876 ETRYTUUI,YTYUIL,UIOKJHGFGH,34567898766 I want to put the numbers ...
0
votes
2answers
70 views

How to sort the data from CSV file by comparing data from text file using awk output it as a TXT file?

There are two files I am going to extract the data: file1.txt: Type Serial ID Element Hit_Possibility Yasuo 19-2 19623 Hasaki file2.csv: Date,Name,Order,Hit Possibility 12-Aug,Ken,1,256 12-Aug,...
-1
votes
3answers
143 views

Field separator part of a column - incorrect parsing unix

I want to do a check the number of columns in a CSV file before processing it. The problem is that the delimiter (comma) also occurs in the text of some fields and because of that I cannot parse it ...
0
votes
2answers
97 views

How to remove duplicate items within the same line from a csv file?

I have a csv file with ~4000 lines, each one containing between 2 and 30 names separated by commas. The names are including titles (for example mr. X Adams or ms. Y Sanders). Some names exist multiple ...
1
vote
3answers
71 views

CSV file: Group and sum values based on first pattern of each value

I am using bash to create a script to find, group and sum fields in a CSV file. Each row will have comma-separated fields with each field following a similar convention. For each comma-separated field,...
3
votes
4answers
582 views

Bash: join data from two csv files

I have two csv files that contain various user data; they share one common field (username). file A: username ; Fullname ; mail Bob ; Bob Hope ; bob@hope.org file B: username ; LastLogonTime ; ...
0
votes
1answer
73 views

Append lines from csv file to another based on column 1 matches

I have 2 .csv files, file1.csv and file2.csv. Say a line from file2.csv has the same 1st field as a line from file1.csv. I would like the full line from file1.csv to replace the full line from file2....
0
votes
3answers
85 views

AWK csv manipulation

I have a csv file containing something like this: abc;!!!!;22 abc;!!!!;23 23;!!!!!;22 Now I want to delte every line that contains a 23 in the third column. Since there is a possibility that there ...
0
votes
2answers
73 views

Rename a csv tab from UNIX bash script

I have CSV file that is generated from a bash script: UNIX_REPORT.sh. When you open CSV the file on the windGrr side with excel, the tab (or sheet) in the lower left corner is named UNIX_REPORT.sh. ...
0
votes
1answer
52 views

Interpolate Columnar Data

I am trying to find a way to interpolate between two lines of data in a CSV file, likely using awk. Right now, each line represents a data point at Hour 0 and Hour 6. I am looking to fill in missing ...
1
vote
3answers
73 views

Using AWK to add a new column with values to a csv file without creating empty new lines in between rows

I would like to add a new column to the end of a csv file and populating the column with values. I have used the following codes and it ends up showing new blank lines in between each record. Please ...
0
votes
2answers
76 views

Split datetime column in csv file into MM/YYYY and time column separately and write to the same csv

I am trying to use awk, split and then print but running into error Input: id day userId itemId 1 12/1/17 8:32 2232 tv 2 1/12/18 10:18 3232 fdfs 3 2/9/18 10:50 232 fdsf ...
-1
votes
3answers
123 views

remove the lines in CSV that contain specific pattern on the second field

How to remove the lines in .csv file that second field contain the word content? example : ams-hbase-log4j,content:\n#LicensedtotheApacheSoftwareFoundation(ASF)underone\n#...
1
vote
2answers
35 views

How to merge different columns in the same file?

I have three files, as shown below: FILE 1: DATE PGTO_CRED 20180801 50.00 20180802 150.00 20180803 130.00 20180804 110.00 20180805 200.00 FILE 2: DATE PGTO_TOTAL 20180801 ...
1
vote
1answer
34 views

Save each line of a .csv file as a separate .csv [duplicate]

I have a .csv file with no headers and want to have each line saved as a separate .csv file with the new file name to be the value in column 2.
0
votes
0answers
42 views

file command gives incorrect encoding type

I have a csv file in hand. When I ran 'file -i filename', it shows that it's encoded as us-ascii. But when I ran cat filename | csvcut -t -e us-ascii, I got an error: "Your file is not "us-ascii" ...
-2
votes
1answer
42 views

Store the content of a txt file into a variable with comma separated [closed]

Hi I have a file txt file with keys a.txt: 1234 4567 4568 4564 6754 warning message invoke from function I need to write the content of this file into a variable parm as 1234,4567,3456,2345 I ...
2
votes
1answer
71 views

Populate a CSV file from data file with nested loops in bash

I have a file, 4.txt that contains full paths to *.cfg files as well as additional data I need to strip for the final report (5.csv). For example /source/EDDG/env1/dom1/proj/config/test.cfg <...
1
vote
3answers
60 views

bash: assign variable and print to stdout in same command [closed]

I'm working on a bash script that reads a csv file with fields and field values and creates a json string out of the input. For simplicity I have a bash script that takes in two arguments simulating ...
-1
votes
3answers
117 views

remove n columns from CSV file [closed]

I have a csv file with data as below : 12,1234,28-07-2018 05:28:12.21,ABC60,3456,28-07-2018 05:28:12.21,AB60,7580,28-07-2018 06:28:12.21,PQ I need to remove all date columns from the file through ...
1
vote
1answer
44 views

Grep regular expression to show only serial numbers (defined length containing Alpha & Digit) in CSV File

I have a lot of (huge and chaotic) CSV files with a bunch of (Cisco) serial numbers. My goal is to extract them (BTW: And to call the Cisco API later on for getting service/support coverage feedback)...
2
votes
4answers
65 views

Move files into separate directories based on grouping defined in a CSV file

RELATED: Move all files with matching prefixes to folder based on a csv list I have a two-column CSV file (comma-separated values), with headers: "id","group" "F1256","old" "E51651","new" "X56369","...
-2
votes
3answers
97 views

Change all but first ',' to “<COMMA>” for each row in file (bash) [duplicate]

I am using bash and have a csv file (dat.csv) that needs to be only two columns (App, Blurb) of data but due to many ',' on each row it is becoming MANY columns. EXAMPLE OF PROBLEM csv.dat: App , ...
4
votes
3answers
278 views

How to print the count of pattern at each line?

I need to print the count of a matching string at the end of each line. An example for matching foo: foo,bar,foo,foo bar,foo,bar,bar foo,foo,bar,bar Result : foo,bar,foo,foo,3 bar,foo,bar,bar,1 ...
-3
votes
1answer
43 views

Script for changing CSV into Key Value (KV) format

I have a CSV file with data as below: row_identifier,DBNAME,tblsps_name,Cur_size,Max_size,Used,Free,Percentage tablespace,MRETF,RERETOSB15_DATA,51200,45600,14284,31316,31 tablespace,MRETF,...
0
votes
4answers
81 views

How to compare lines in a single file line by line and append the output to the same file

I have a file abc.csv that has the below contents. a6caa290d03ae75dd5f3bf9214f0b523;CO.jar,a6caa290d03ae75dd5f3bf9214f0b523;CO.jar, a1a5bf1547f50feea4120290343f89e4;Common.jar,...
0
votes
0answers
77 views

Reading CSV and writing CSV with Base64 encoding using cat

So I been trying to make image encoding script what I got working with 3 columns but not with 4. Example data is just with 4 rows finished data will be 100-1000 rows... Delimiter on csv is set to | ...
1
vote
1answer
142 views

Switch columns in CSV using awk? [duplicate]

I have the exact same question as this one here. Only difference is I have 3 columns only and I am trying to move the 1st column to the end. The original file looks like this: col1,col2,col3 2,2015-...
5
votes
3answers
375 views

sed/awk to remove double quotes " that are inside of curly braces {}

I have a csv file with data similar to this: "1b579a5e-9701-40eb-bd36-2bc65169da99","week14_Friday-019","6907eaad-1aff-4d26-9088-ba20374b67c0","2181-019","f20af5bb-c716-42e0-9b9d-cbolf5bfecea","15-...
-1
votes
3answers
267 views

Bash to read all columns and display blank cells in each row from CSV file

I need the script to go through each column and check for blanks and group it up for me into 1 column... so e.g. 1st row: banana , 2nd row: rachel, 5th row: rachel, monica, chandler, joey, janice, ...
0
votes
1answer
28 views

How to do sum calculations in libreoffice calc, single column

I have a .csv file which has two columns and about 50 rows of data in them. One of the columns has some numbers in it. What should have worked is - =sum<B1:B50> the result should be displayed ...
1
vote
2answers
647 views

Using bash to swap first and second columns in CSV

I'm using bash. I have a CSV file with two columns of data that looks roughly like this num_logins,day 253,2016-07-01 127,2016-07-02 I want to swap the first and second columns (making the date ...
2
votes
1answer
51 views

Merge multiple lines column by column based on column 1 - same file?

I'm trying to organise a CSV file with 30 columns and 1000 rows. I want to concatenate the rows column by column if the 1st column is the same, using a vertical line (|). Input: 3,XX,YY,123,1234,MM,-...
4
votes
3answers
240 views

How to merge first two lines of a csv column-by-column?

I have an excel file that I converted to csv. When converted, it looks like the following example (Please note that there are 100+ columns in the csv. This is a minified version): ,Product," ",Citty,...