1

Grep/Awk/Sed for a set of lines from "0010|" to "0070|" AND that contains a match in the $PH_NO

Below is the sample data. I need to grep for Phone number that present in 0012 field and the corresponding complete customer records, lines from 0010 to 0070. One data file may contain two or three customer records with same phone number and I need to get all of them.

0010|Kumar||57 Rich street|Chennai|Tamil Nadu|
0011|20171115| ID
0012|149 196 222| PH Number 
0013|20161101|20171102|
0022|Payment Method |Lucky customer|
0080|P|5.00-|20161111|Payment|
0080|P|5.00-|20161130|Payment|
0080|TP|10.00-|||
0070|000AYDCHDFF|820|762|
0010|RAM||57 Rich street|Chennai|Tamil Nadu|
0011|20171115| ID
0012|149 196 333| PH Number 
0013|20161101|20171102|
0022|Payment Method |Lucky customer|
0080|P|5.00-|20161111|Payment|
0080|P|5.00-|20161130|Payment|
0080|TP|10.00-|||
0070|000AYDCHDFF|820|762|
0010|Joe||57 Rich street|Chennai|Tamil Nadu|
0011|20171115| ID
0012|149 196 222| PH Number 
0013|20161101|20171102|
0022|Payment Method |Lucky customer|
0080|P|5.00-|20161111|Payment|
0080|P|5.00-|20161130|Payment|
0080|P|5.00-|20161111|Payment|
0080|P|5.00-|20161130|Payment|
0080|P|5.00-|20161111|Payment|
0080|P|5.00-|20161130|Payment|
0080|TP|10.00-|||
0070|000AYDCHDFF|820|762|

Note: I'm on an AIX server with ksh.

  • 2
    It's not clear if $START, $END and $MIDDLE are line numbers or patterns. – Kusalananda Nov 24 '17 at 7:59
  • XY problem. It would be easier and cleaner with a perl -wpe or python that the old tools you suggest. Also edit the question to clarify if 0080 should be printed or not. Also clarify if 0070 marks end of record. – kubanczyk Nov 24 '17 at 8:11
  • 1
    Edited my requirement. Hope this time it's clear. Please suggest. – Santhosh Ram Nov 24 '17 at 8:43
  • 1
    Please give a desired output showing which lines from your sample should be returned :) – Zanna Nov 24 '17 at 9:33
3

So you want the whole record from 0010 to 0070, if the PH Number matches $PH_NO"? Then this sed oneliner will work:

sed "/^0010/,/^0070/H;/^0010/h;/^0070/! d;x;/|$PH_NO| PH Number/! d"
  • /^0010/,/^0070/H to append one record from 0010 to 0070 to the hold space
  • /^0010/h the 0010 shouldn't get appended, but start a new record, so copy it to the hold space
  • /^0070/! d no further processing or output unless it's the 0070 item
  • x;/|$PH_NO| PH Number/! d" exchange the spaces, so the whole record is in the pattern space now and delete it if it doesn't contain the said number.
  • simpler answer than mine. vote up. – Bruce Nov 24 '17 at 11:44
  • @Philippos, I'm trying the same for a different file now. but getting the below error. Im still on same AIX. Could you suggest? sed '/^12/,/^16/H;/^12/h;/^16/! d;x;/148 558 349/! d' AWBK_00R.txt sed: 0602-403 /^12/,/^16/H;/^12/h;/^16/! d;x;/148 558 349/! d is not a recognized function. – Santhosh Ram Feb 15 '18 at 8:28
  • Do you have an idea where this 0602-403 comes from? – Philippos Feb 15 '18 at 8:57
  • Not sure. At least not from what i type. But the same command working fine in bash. :( – Santhosh Ram Feb 15 '18 at 9:12
  • bash on that AIX machine with the same sed? Or a different sed version, perhaps GNU sed. In the latter case please try to reproduce it with option sed --posix '... Otherwise split the file to see whether there is a certain record that produces the problem. – Philippos Feb 16 '18 at 11:26
0
for r in `grep -n '^0010\|^0012\|^0070' CUSTOMER_FILE | grep -C1 '[0-9]\+:0012|149 196 222|' | grep -o '^[0-9]\+' | paste -d, - - - | sed 's/,[0-9]\+,/,/g'`; do sed -n "$r"p CUSTOMER_FILE; echo; done

149 196 222 in above command is the customer phone number. Change it to the phone number you want to find out.

CUSTOMER_FILE is the file you want to search. Change it to your file name.

You also can put the code in a bash script, and replace 149 196 222 with $1 and replace CUSTOMER_FILE to $2. Say find-customer.sh, then you can execute the script like this

./find-customer.sh '149 196 222' your-file-name

Prerequisite of this piece of code:

  1. Bash, GNU environment (GNU grep, GNU sed)
  2. Your file must follow the format like below 0010 ... <no 0010 or 0012 or 0070> ... 0012 ... <no 0010 or 0012 or 0070> ... 0070 ... <repeated content as above or end of file>


UPDATE

This is a high-performance version. (At least higher than the original one above. No for-loop involved at all.)

grep -n '^0010\|^0012\|^0070' CUSTOMER_FILE | grep -C1 '[0-9]\+:0012|149 196 222|' | grep -o '^[0-9]\+' | paste -d, - - - | sed -r 's|([0-9]+),[0-9]+,([0-9]+)|\1,\2p;\2a|g' | sed -n -f - CUSTOMER_FILE


UPDATE for AIX

Since the asker is working on AIX. The grep on AIX doesn't support context options, -A, -B, -C.

On Internet, there are a variety of "cgrep" (context grep) implementations to resolve this problemd (to simulate GNU grep context option). But most of them can't give as same output as GNU grep. Only one I found is the closest to the GNU grep context option. The link is https://stackoverflow.com/questions/1685678/advanced-grep-unix/1685782#1685782

I made some necessary modifications for this case.

#!/bin/bash
BEFORE=$1
AFTER=$1
FILE=/tmp/.cattmp
PATTERN="$2"
cat > $FILE
for i in $(grep -n "$PATTERN" $FILE | sed -e 's/\:.*//')
  do head -n $(($AFTER+$i)) $FILE | tail -n $(($AFTER+$BEFORE+1))
done
rm $FILE

Save this file as grep-context.sh and replace grep -C1 with ./grep-context.sh 1 in my command above.

Another way I think is once and for all is to compile GNU grep on AIX. (also compile GNU sed just in case)

  • sorry about that. I forgot to mention that im on AIX. Getting the below error. grep: Not a recognized flag: C grep: Not a recognized flag: 1 Usage: grep [-r] [-R] [-H] [-L] [-E|-F] [-c|-l|-q] [-insvxbhwyu] [-p[parasep]] -e pattern_list... [-f pattern_file...] [file...] – Santhosh Ram Nov 24 '17 at 10:11
  • That's why I clarified the prerequisite of my answer. I knew somebody may not work in GNU environment. For AIX grep, there is no context options, -A, -B and -C. I have updated my answer. Hope it can help you. Actually I suggest you compile GNU grep and GNU sed to avoid any further problems. – Bruce Nov 24 '17 at 11:41
-1

You can use this script:

#!/bin/sh
read START
read END
read MATCH

REND=$(grep -n "$END" lines | tail -1 | cut -d":" -f 1)
RSTART=$(grep -n "$START" lines | head -1 | cut -d":" -f 1)

sed $RSTART,$REND!d lines | grep "$MATCH"

Put it in a file and add the execute permission with this command

chmod +x script.sh

START variable: Your starting line number (e.g 0010)

END variable: Your ending variable (e.g 0070)

MATCH variable: The word/character/number you are looking for in a line (e.g 0012)

RSTART variable: The starting line number in the text file (e.g 1)

REND variable: The starting line number in the text file (e.g 32)


EDIT:

You can also see how many times a line repeated, by changing the last line to this:

sed $RSTART,$REND!d lines | grep "$MATCH" | sort | uniq -c
  • Hi Amir, didn't get the expected output. check the below. SERVER1# sh -x data_retrieve.scr awbil_p_mea_ac1_20171115.aws_023244.aws + read START 0010 + read END 0070 + read MATCH 1483894225 + + grep -n 0070 awbil_p_mea_ac1_20171115.aws_023244.aws + tail -1 + cut -d: -f 1 REND=42977 + + grep -n 0010 awbil_p_mea_ac1_20171115.aws_023244.aws + head -1 + cut -d: -f 1 RSTART=2 + sed 2,42977!d awbil_p_mea_ac1_20171115.aws_023244.aws + grep 1483894225 0023|6338010000000001483894220|1483894225| 12858 12858 1483894225 5|1483894225 V4322251647 91 X| SERVER1# – Santhosh Ram Nov 24 '17 at 10:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.