2

How to print the lines before a search keyword without using grep in a .gz file?

ID:342N000390AAAAAAAA   07/14/15 10:26      (MV90    )
         * Register Data Imported
         * Warning - No Profile Data
     07/14/15 10:24  05/13/15 08:16       15    1   5956

I want to search the No Profile Keyword and print the ID NUMBER THAT IS 342N000390AAAAAAAA and necessarily "No Profile" won't come in third line; it may come in any line. My Operating system is HP-UX, so many commands are not eligible.

  • What's your OS? – cuonglm Jul 24 '15 at 7:45
  • 1
    Can you please run grep --version and try using grep -C2 (context)? – Emeric Jul 24 '15 at 7:46
  • Since they are .gz files you need to use zgrep – jcbermu Jul 24 '15 at 7:51
  • My OS HP-UX aehpdcds01 B.11.31 U ia64 0209983576 unlimited-user license – Pooja Jul 24 '15 at 9:20
  • HPUX don't have the same grep as Linux; so you could build and install GNU grep, perhaps configured with --prefix=$HOME/soft/ – Basile Starynkevitch Jul 27 '15 at 10:36
3

You could work with awk instead:

gzcat file.gz | awk '/No Profile Data/{printf "%s\n%s\n%s\n", b, a, $0} {b=a;a=$0}'
  • gzcat (or zcat on Linux) prints the content of the gzipped file to the standard output
  • awk then searches for the string "No Profile Data" and prints the two previous lines
  • I tried but its not printing the previous two lines – Pooja Jul 24 '15 at 9:52
  • @Pooja Try now, I edited my answer – chaos Jul 24 '15 at 10:11
  • zcat IIRC on HP-UX only supports file.Z (ie compress) and not gzip – DarkHeart Jul 26 '15 at 9:13
  • change zcat to 'gunzip -c' – DarkHeart Jul 26 '15 at 12:19
  • Hi . It showing syntax error in this line { printf "%s\n%s\n%s\n", b, a, $0} – Pooja Jul 27 '15 at 9:15
2

Edited

New approach: removing line breaks.

Assuming that you have only one ID per gzipped file, you can try the following:

gunzip -c file.gz | sed -e ':a;N;$!ba;s/\n/ /g' -e '/^[[:space:]]/d' -e 's/^ID:\([[:alnum:]]*\).*Warning - No Profile Data.*/\1/' -e '/^ID:/d'
  • gunzip -c extracts the file to stdout
  • sed collapses all lines into one, then removes all lines not starting with ID:, then extracts the ID from matching files, then removes the ID: line for non-matching files, so that it displays either the ID or nothing.

Credits due to https://stackoverflow.com/a/1252191/5148242 and https://unix.stackexchange.com/a/218094/124507 @DarkHeart

Original

grep is still the appropriate option, but for the sake of the exercise, you can use sed for the finding, and paste for the -B2 part:

zcat nogrep.gz | paste - - - | sed -e '/^[[:space:]]/d' -e 's/^ID:\([[:alnum:]]*\).*Warning - No Profile Data/\1/' -e '/^ID:/d'
  • paste joins each group of 3 lines
  • sed removes all lines not starting with ID:, then extracts the ID from matching files, then removes the ID: line for non-matching files, so that it displays either the ID or nothing.
  • can any one suggest how to solve this since OS does not support this commands – Pooja Jul 26 '15 at 4:31
  • Did you try the updated version from @chaos? – Emeric Jul 26 '15 at 4:43
  • Yes its showing error in syntax – Pooja Jul 27 '15 at 9:03
  • 1
    @Emeric - in that case you could save yourself a lot of effort (and many possible misses) by doing sed -ne'/^ *ID/!d;N;N;/No Profile/s/ .*//p'. As is you're bound to miss any match if the sequence goes off - for example if there is a group of 4 lines in one group and 3 in another. – mikeserv Jul 27 '15 at 11:50
  • 1
    @mikeserv - Yes, this one I actually tested. As someone else pointed out, there is gzcat. zcat file.gz file.gz.Z: No such file or directory – DarkHeart Jul 27 '15 at 23:57
2

(hopefully) the final product


find . -name \*.gz -type f -exec   gzcat {} +  |
sed -ne'/^ *ID:/h;/No Profile/!d;x' \
    -e's/^ *ID:\([^ ]*\).*/\1/p'

So that will recursively find all of the regular files rooted in the current directory with filenames which match the pattern *.gz and call zcat as few times as is necessary to iteratively uncompress every one in a single stream to sed's stdin.

sed will scan its input for lines which are headed with the string *ID:. It will hold a copy if found, and next look for No Profile while deleting all lines which don't match. When found sed will swap into hold space and try to trim an ^ *ID: line to only the portion falling between the first : and next occurring <space>. If successful, sed prints the results.

As @DarkHeart points out you will most likely have to alter the zcat command name to gzcat on an HPUX system, though.


variations


This would be all you need to search a single file for line pairs occurring immediately previous to a match for the string No Profile:

gzip -d <file.gz |
sed -e'1N;$!N;/\n.*No Profile/P;D'

That will just scan input three lines at a time. Each line is separated by a \newline in pattern space. As each New line is pulled in, the oldest is Deleted. If the regexp \n.*No Profile is ever matched in pattern space (as it will be when it is the newest line in pattern space, and the following cycle when it is the second to newest), the oldest line is printed. And so you get the two lines occurring before No Profile. If you want to also print the line on which it is found...

gzip -d <file.gz |
sed -e'1N;$!N;/No Profile/P;D'

with find:


find . -name \*.gz -type f -exec zcat {} + |
sed -e'1N;$!N;/No Profile/P;D'

You can swap the . there for the name of some directory if you like. You can also add the \n.*No Profile bit to avoid printing the matching line. That command will recurse into all child directories of .. If that is not to your liking:

find . \! -name . -prune -name \*.gz  \
          -type f -exec zcat {} +     |
sed -e'1N;$!N;/No Profile/P;D'

If you are specifically looking for the leading ID field and only if can be found two lines prior to a match for No Profile you can do:

find . -name \*.gz -type f -exec zcat {} + |
sed -ne'/^ID/!D;/\n/!N;N' \
     -e's/ .*\n.*\n.*No Profile.*//p;D'

...which would print only the leading ID field as it might occur in any/all of the *.gz files find calls zcat to print and only if ID definitely occurs two lines before a No Profile match.

  • Comments are not for extended discussion; this conversation has been moved to chat. – terdon Jul 28 '15 at 7:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.