0

I have a CSV that's separated like so with no header:

epochtime,#value,#value,property=1.property=2.property=3 

The individual properties are separated by periods but are contained within a single column in the csv.

I am trying to run a blacklist with a list of properties to filter through the specific properties. I am using this grep function below:

grep -vFf blacklist.txt file.csv > newfile.csv

However it returns with no results. When I remove the other values and epoch time, it works perfectly leading me to suspect the periods might not be an issue.

Is there a way I can ignore the other two columns and still have it return the proper results?

Thanks in advance. I am quite new to unix. Would an awk command be better suited?

  • 1
    The periods shouldn't be an issue, since you are using the -F (fixed strings) flag. Can you post a specific minimal example of a file.csv and a blackist.txt that demonstrate the issue? – steeldriver Jan 6 '17 at 0:04
  • As steeldriver said, with a short example of the contents of the blacklist, we can most likely figure out what's not working properly. – Kusalananda Jan 27 '17 at 23:07
0

If anyone stumbles into this post via search looking for an answer. I wrote a small python script instead.

import csv 
import os

blacklist_dict = {}

with open("blacklist.txt", 'r') as blacklist:
   for line in blacklist:
    line = line.strip('\n')
    blacklist_dict[line] = 0
blacklist.close()


with open('filename.csv', "r") as source_file, open('newfile.csv', "w") as target_file:
reader = csv.reader(source_file)
writer = csv.writer(target_file)
for row in reader:
    if row[2] not in blacklist_dict:
        writer.writerows([row])

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.