Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a file of strings in each line.

For example:

AAAAA   
BBBBB   
CCCCC  
etc  

I want to check each string one by one whether it exists in a series of files in a specific directory.

I know that I can do: grep -f filename to do the search. But what I need is to use one more regular expression per line to get more specific results. The file with the strings is big and changes often so it does not make sense to modify each line to add an expression.

So what I need is to combine the grep -f filename to loop over each line of the file (and grep for it in the files) but add some more conditions. E.g. that AAAAA exists and [0-9]{1,3}$ exists as well in the same line.

share|improve this question
    
Is the additional condition different for each line of the file? –  Joseph R. Aug 21 '13 at 21:00
    
@JosephR.:No.The same condition for all lines of the file.But the file's contents are changing constantly and are a lot so I need to search for these contents.But the condition X would be the same for all lines –  Jim Aug 21 '13 at 21:02
add comment

2 Answers 2

You can do this stepwise with grep and a pipeline

grep -f your_file list_of_files|grep -E extra_condition

or

while read line; do
    grep -E -e "$line" -e extra_condition list_of_files
done < your_file

In both cases, the -E switch is needed if you're going to use a pattern of the form [0-9]{1,3}$ (i.e. containing numerical quantifiers).

Edit

Thanks to Drav Sloan's comment, we know that the second version (the while loop), not only can be reduced to

grep -E -e "extra_condition" -f your_file list_of_files

but that it actually looks for one pattern or the other not one pattern and the other. You should stick with the first form.

share|improve this answer
    
So this passes one line of the file at a time to grep -E? So something like: grep -f filename ./*.csv |grep -E regex? –  Jim Aug 21 '13 at 21:05
    
@Jim Basically, yes. –  Joseph R. Aug 21 '13 at 21:06
    
Sorry for the lamen question.The second script is exactly as you post it? I.e. redirect my file to done? –  Jim Aug 21 '13 at 21:08
    
@Jim Yes, the redirection actually affects the read call which should be expecting input on stdin and you make it read from the file instead. –  Joseph R. Aug 21 '13 at 21:09
2  
you can reduce the while loop version to grep -E "extra_condition" -f your_file list_of_files. Note this has the effect of OR rather than AND like the first example. –  Drav Sloan Aug 21 '13 at 22:32
show 2 more comments

Just to show an altnerative way using awk and grep.

Sample Data

$ cat patterns.txt 
AAAA
BBBB
CCCC
DDDD
EEEE

$ cat sample.txt 
AAAA 1
AAAA 2
AAAA 3
AAAA 4
DDDD 1
DDDD 2
YYYY 1
YYYY 2

Examples

awk #1:

$ awk 'FNR==NR{a[$0]; next}($1 in a)' patterns.txt sample.txt | grep -E "[0-9]{1,3}$"
AAAA 1
AAAA 2
AAAA 3
AAAA 4
DDDD 1
DDDD 2

This will most likely need to be adjusted but gives you a hint as to how to do it.

awk #2:

$ awk 'FNR==NR{a[$0];next};{for (i in a){if (match($0,i)) print}}' patterns.txt sample.txt | grep -E "[0-9]{1,3}$"
AAAA 1
AAAA 2
AAAA 3
AAAA 4
DDDD 1
DDDD 2

This one is more flexible, it can handle the pattern strings being anywhere within the line of the sample data.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.