0

I have a CSV that's separated like so with no header:

epochtime,#value,#value,property=1.property=2.property=3 

The individual properties are separated by periods but are contained within a single column in the csv.

I am trying to run a blacklist with a list of properties to filter through the specific properties. I am using this grep function below:

grep -vFf blacklist.txt file.csv > newfile.csv

However it returns with no results. When I remove the other values and epoch time, it works perfectly leading me to suspect the periods might not be an issue.

Is there a way I can ignore the other two columns and still have it return the proper results?

Thanks in advance. I am quite new to unix. Would an awk command be better suited?

2
  • 1
    The periods shouldn't be an issue, since you are using the -F (fixed strings) flag. Can you post a specific minimal example of a file.csv and a blackist.txt that demonstrate the issue? Commented Jan 6, 2017 at 0:04
  • As steeldriver said, with a short example of the contents of the blacklist, we can most likely figure out what's not working properly.
    – Kusalananda
    Commented Jan 27, 2017 at 23:07

1 Answer 1

0

If anyone stumbles into this post via search looking for an answer. I wrote a small python script instead.

import csv 
import os

blacklist_dict = {}

with open("blacklist.txt", 'r') as blacklist:
   for line in blacklist:
    line = line.strip('\n')
    blacklist_dict[line] = 0
blacklist.close()


with open('filename.csv', "r") as source_file, open('newfile.csv', "w") as target_file:
reader = csv.reader(source_file)
writer = csv.writer(target_file)
for row in reader:
    if row[2] not in blacklist_dict:
        writer.writerows([row])

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .