2

I'm trying to find just the files' names which contain an specific string. The files are compressed (.gz).

I don't have zgrep installed and can't install it. Therefore I can't use the -l option.

I've tried using gzip and gunzip with the -c option and pipping to grep -l but that did not work, I have also used zcat but that also did not work. Any clue?

(Note: the OS is Solaris 10).

  • is it a tar.gz file? – Rob Oct 21 '15 at 19:19
  • 2
    What is mean is it one tar.gz with multiple files in it or each file is gziped separately? – Rob Oct 21 '15 at 19:53
  • Hi Rob, yes the files are ".tar.gz" and all are individual files. (All are compressed logs). Thanks. – Luis Oct 22 '15 at 12:47
  • From your grep: illegal option -- q Usage: grep -hblcnsviw pattern file, you're on Solaris 10 or earlier right? – Stéphane Chazelas Oct 23 '15 at 15:05
  • Hi @StéphaneChazelas, yes, you are right, it's a Solaris 10 (5.10). – Luis Oct 23 '15 at 15:58
3

You can do the work of zgrep manually. Since you only want the file names, use grep just to test the presence of the pattern, and print out the file name if the pattern is found.

#!/bin/sh
pattern=$1; shift
PATH=`getconf PATH`:$PATH # needed on Solaris 10 and earlier
                          # to get a standard grep
export PATH
found=0
for x do
  if case "$x" in
      *.gz|*.[zZ]) <"$x" gzip -dc | grep -q -e "$pattern";;
      *) <"$x" grep -q -e "$pattern";; 
     esac
  then
    found=1
    printf '%s\n' "$x"
  fi
done
if [ $found -eq 0 ]; then exit 1; fi

To be run as:

that-script 'pattern' file1 file2.gz file3.Z file.*.gz ...

A few notes specific to you running Solaris 10 (also applies to earlier versions and in some respects to Solaris 11 as well).

  • on those systems, /bin/sh is a Bourne shell as opposed to a standard POSIX sh. You've got the choice of either changing your she-bang to #! /usr/xpg4/bin/sh - to get a standard sh, or restrict yourself to the ancient Bourne syntax like we do here (so no $(...), no case $x in (x)...) (Solaris 11 is now using a POSIX compliant shell for its /bin/sh (ksh93)).
  • on those systems, zcat only handles .Z files as compressed by compress as they were in the olden days. You need to invoke gzip for .gz files.
  • By default, you don't necessarily get standard utilities. For instance, the default grep in /usr/bin is an ancient one that doesn't support the standard -q option. To get the standard utilities, you need to update $PATH with the paths where to find the standard utilities (as output by getconf PATH).

If you want to display both the archive member name and the line number or content, you'll need to get the line data from grep and the member name from the script. Remove the -q option from the grep invocation, and postprocess its content.

#!/bin/ksh
pattern=$1; shift
export PATH="$(getconf PATH):$PATH" # needed on Solaris 10 and earlier
                                    # to get a standard grep
found=0
for x do
  case "$x" in
      *.gz|*.[zZ]) <"$x" gzip -dc | grep -n -e "$pattern";;
      *) <"$x" grep -n -e "$pattern";; 
  esac | {
    filename=$x awk '{print ENVIRON["filename"] ":" $0; found=1}
                     END {exit(!found)}' && found=1
  }
done
if [ $found -eq 0 ]; then exit 1; fi
  • zcat -f and avoid the case – roaima Oct 21 '15 at 20:19
  • @roaima That also treats non-.gz files differently, in particular that will search through the first file of a zip file only. I'm not sure that's desirable, I think I'd prefer text in zip files to be skipped altogether than found only if it's in the first member. I could add a case for .zip files, but zcat -f doesn't scale that way. – Gilles Oct 21 '15 at 20:25
  • Oh ok. I'd not considered zip files as they weren't mentioned; just gz and plain. – roaima Oct 21 '15 at 20:27
  • Thanks @Gilles, where am I supposed to add the patter and file information (like *.gz). – Luis Oct 22 '15 at 17:48
  • @Luis I don't understand the question. Do you want to add support for other file types? – Gilles Oct 22 '15 at 18:01
1

Here's another solution:

#!/bin/bash
# Grab the pattern, just like grep
re="$1"; shift

# Loop across the remaining arguments, or stdin if none
test 0 -eq $# && set -- -
for file
do
    # Search through the file (compressed or otherwise)
    zcat -f "$file" | grep -q "$re" && echo "$file"
done

You should copy this into a file and make that file executable (chmod a+x {filename}, and then use it similarly to zgrep:

mkdir -p ~/bin
export PATH="$HOME/bin:$PATH"  # Add the ~/bin directory to your PATH. Also add this line to your ~/.profile

cat > ~/bin/mygrep             # Paste the file at this point, hit Ctrl/D on a blank line to end. Or otherwise edit ~/bin/mygrep
chmod a+x ~/bin/mygrep         # Make it executable

mygrep 'hello.*world' a*       # Search for the RE 'hello.*world' in all files beginning with 'a'

If you find that zcat -f or grep -q aren't recognised you can swap each for the following

( zcat "$file" 2>/dev/null || cat "$file" )    # zcat -f "$file"
grep "$re" 2>/dev/null                         # grep -q "$re"

With both substitutions applied the resulting replacement line would look like this

( zcat "$file" 2>/dev/null || cat "$file" ) | grep "$re" 2>/dev/null && echo "$file"
  • Hi Roaima, thanks for your answer. Where am I supposed to place the file and pattern information?, Like this? re="$the_desired_pattern"; shift and for *.gz – Luis Oct 22 '15 at 17:41
  • @Luis, no. you use it like you would use zgrep. I'll update the question with the information. (The same goes for Gilles' answer.) – roaima Oct 22 '15 at 18:15
  • Thanks @roaima, it looks like my grep and gzip version do not support some of the options in the script, Invalid Option Usage: zcat [file ...] grep: illegal option -- q Usage: grep -hblcnsviw pattern file . . . – Luis Oct 22 '15 at 18:37
  • Thanks @roaima for taking the time. This has been quite helpful. – Luis Oct 26 '15 at 18:17
  • @Luis I've updated the answer with possible alternatives for you – roaima Oct 26 '15 at 19:26
0

Since your files are actually tar.gz file you need to untar them as well. Tar is a archiving utility that packages multiple files together (in your case only one) and was originally used to backup data to tape drives.

The tar utility did not originally compress content hence why it was usually streamed through gzip or other utilities for compression.. if you unzip a tar.gz you are still left with the tar achieving layer which is why it is not working.

these days gun tar will tar and zip for you in one command using the "z" command.

so in order for you to do what you want I think you need to do use the tar extract "x" unzip "z" from a file target "f" and stream to stdout "O" then pipe to grep. It should look something like this:

 tar -xzf mycompressedlogfile.tar.gz -O | grep -l "pattern"

Since your tar version does not have -z then try piping the output of gzip through tar before your grep

 gzip -dc mycompressedlogfile.tar.gz | tar -xOf - | grep -l "pattern"

But I do not think Solaris 10 version of tar support the "O" option to stream the tar content to stdout. :( You are probably going to have to do the above (without the O so "tar -xf -") for each file from within a working directory, or some way to make sure you can account for the files that are created, search the content, and then delete the file(s). Sorry unless you can install gnu-tar on the Solaris machine.

I guess from the answer selected you were able to search the tar stream for the pattern (since it is not compressed I guess that makes sense) and figure it out from there.. :) good deal.

Glad you got your answer.

  • Hi @Rob , thanks for the information. Unfortunately the deployed tar version is an old one. I does not have the -z option available. – Luis Oct 23 '15 at 15:30
  • I updated my post, but I think you are not going to be able to do it since from the Solaris man page for tar that I found, the "O" option is not available so you will not be able to stream the content of the tar to standard out. At least I do not know of a way. – Rob Oct 23 '15 at 21:12
  • Thanks @Rob for taking the time. Yes, the tar version deployed on the server does not have the O option. – Luis Oct 26 '15 at 18:07
  • You are most welcome.. I am glad someone got you an answer that worked out :). Even it was not me. – Rob Oct 27 '15 at 12:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.