I have two files and I want to grep the content of fileA(list of phone numbers) one-by-one from multiple files fileB fileC fileD and print the first match of each number. I want to print each phone number after the first hit and grep the next until the last phone number. Note if number is already found in file B no need to grep C and D just move to next number. Please help

I was using this before
zgrep -f number_list.csv voice*20170301*gz | awk -F "|" '{print $22}' >> register_20170301.csv
zgrep -f number_list.csv sms*20170301*gz | awk -F "|" '{print $22}' >> register_20170301.csv
zgrep -f number_list.csv data*20170301*gz | awk -F "|" '{print $22}' >> register_20170301.csv

  • 2
    better give a sample of the files and the output you want - is there anything else in the files? – Zanna Mar 1 '17 at 7:37
  • 1
    Your second sentence is confusing and seems to contradict the clear statement in the first sentence. If you find the first phone number in fileB should we restart the search for the next phone number from fileA? – roaima Mar 1 '17 at 9:12
  • Approximately how many lines does each file contain? (Tens? Hundreds? Hundreds of millions?) If the log files are significantly larger than the list of desired phone numbers you may gain efficiencies by scanning all the log files just once, and spending effort on parsing the results to print only the first of each pattern match. – roaima Mar 1 '17 at 9:21
while read phone_number
    for file in data*20170301*gz
        zgrep -q "${phone_number}" "${file}"
        if [ "$?" -eq "0" ]
             zgrep -q "${phone_number}" ${file} >> register_20170301.csv
done < number_list.csv
  • cool +1 but zgrep -q ... >> f will write nothing; perhaps zgrep -m 1 ...? – JJoao Mar 1 '17 at 8:53
  • @Kamaraj This is working when I substitued -q with -m 1 as suggested by JJoao but it looks like the code grep only data*20170301*gz file. I want it to grep from voice*20170301*gz if nothing is found in data*20170301*gz and move to sms*20170301*gz if nothing is found in voice*20170301*gz – yemmy Mar 1 '17 at 9:12
  • and I want to print just column $22 and not the whole file – yemmy Mar 1 '17 at 9:32
  • @StéphaneChazelas – yemmy Mar 1 '17 at 15:39

With grep, using the -m switch will allow you to stop reading a file after X number of lines. For example grep -m 2 will stop after finding 2 matches. In your case you would use grep -m 1 to stop after the first match. A grep command that find a match will return a successful exit code while a grep command that finds no matches exist with a non-zero. You can use if/then/elif/fi or [[ test ]] blocks to prevent the latter grep commands from running if the first grep returns a result successfully.

TEMP=`zgrep -m 1 -f number_list.csv voice*20170301*gz`
[[ "$TEMP" ]] || TEMP=`zgrep -m 1 -f number_list.csv sms*20170301*gz`
[[ "$TEMP" ]] || TEMP=`zgrep -m 1 -f number_list.csv data*20170301*gz`
echo "$TEMP" | awk -F "|" '{print $22}' >> register_20170301.csv
  • cool (+1) but it will not give the information about all the number_list. – JJoao Mar 1 '17 at 8:47

If files are not too big:

zgrep -f number_list.csv {voice,sms,data}*20170301*gz  | 
   awk -F'|' '{print $22}' | sort -u

Not the answer you're looking for? Browse other questions tagged or ask your own question.