Grep command in shell script

Question

I've been writing a shell script which should make grep command outputs for further use. However when I pass a variable containing ~/.../multiple_dir/* to grep as input I get empty output file.

#!/bin/sh

set -u

PROGRAM="$1"
REGEXP=$(cat "$2")
INP_FILE="$3"
OUT_FILE="$4"

printf "%-30s: " $(basename ${INP_FILE})

if [ $INP_FILE = "STDIN.inp" ]
then
    cat ${INP_FILE} | ${PROGRAM} ${REGEXP} - > ${OUT_FILE} 2>&1
elif [ $INP_FILE = "MULTIPLE.inp" ]
then
    ${PROGRAM} ${REGEXP} '$(cat ${INP_FILE})' > ${OUT_FILE} 2>&1
else
    ${PROGRAM} ${REGEXP} ${INP_FILE} > ${OUT_FILE} 2>&1
fi

true

This is the script I've written, with single files or from STDIN it works like a charm, but when $(cat ${INP_FILE})=~/.../multiple_dir/* it doesn't work.

I have ./inputs, ./inputs/multiple_dir/, and ./tests directories. In ./tests directory, I have a link to ./inputs directory files, which are SINGLE (pdb file), STDIN (pdb file which I will pass as STDIN), UNREADABLE (no read bit), MULTIPLE (contains ~/.../inputs/multiple_dir/*) and EMPTY (empty).

I have a Makefile which calls shell script and passes one file at a time as a parameter. Each case works perfectly except when it passes MULTIPLE, then I get an empty output file. In all other cases output files aren't empty. The values makefile passes to script is as follows grep regExp/regExp(contains text ATOM) tests/MULTIPLE.inp(contains text ~/.../inputs/multiple_dir/*) outputs/MULTIPLE.out

To better illustrate the problem I'm experiencing i've wrote a few short scripts.

#!/bin/sh
grep ATOM ~/5as-darbas/inputs/multiple_dir/* > working.out

#!/bin/sh
input_file="$1"
#$1 is a file containing text ~/.../inputs/multiple_dir/*
echo "$(cat ${input_file})"
grep ATOM "$(cat ${input_file})" > not_working.out

Scripts results are as follows:

simas@Lenovo:~/5as-darbas/inputs$ ./working 
simas@Lenovo:~/5as-darbas/inputs$ ./not_working MULTIPLE
~/5as-darbas/inputs/multiple_dir/*
grep: ~/5as-darbas/inputs/multiple_dir/*: No such file or directory
simas@Lenovo:~/5as-darbas/inputs$ ls -l
total 2372
-rw-rw-r-- 1 simas simas       0 Dec 10 19:37 EMPTY
-rw-rw-r-- 1 simas simas      35 Dec 12 02:49 MULTIPLE
-rw-rw-r-- 1 simas simas      25 Dec 12 02:43 MULTIPLE~
drwxrwxr-x 3 simas simas    4096 Dec 12 01:52 multiple_dir
-rwxr-xr-x 1 simas simas     163 Dec 12 03:26 not_working
-rw-rw-r-- 1 simas simas     161 Dec 12 03:26 not_working~
-rw-rw-r-- 1 simas simas       0 Dec 12 03:29 not_working.out
-rw-rw-r-- 1 simas simas       0 Dec 12 01:52 s~
-rw-rw-r-- 1 simas simas  332343 Dec 10 19:38 SINGLE
-rw-rw-r-- 1 simas simas  252720 Dec 10 19:38 STDIN
--w--w---- 1 simas simas  252720 Dec 10 19:38 UNREADABLE
-rwxr-xr-x 1 simas simas      70 Dec 12 03:27 working
-rw-rw-r-- 1 simas simas     128 Dec 12 03:27 working ~
-rw-rw-r-- 1 simas simas 1554999 Dec 12 03:29 working.out

Note the size of working.out and not_working.out.

I was able to get this script to work. The thing is when you pass a path ~/path/to/somewhere/* shell for some reason can't find it, however if you pass path like this /home/name/path/to/somewhere/* it works like charm.

Answer 1

Try using eval in your statement. Eval will evaluate the value of $INP_FILE before cat'ting them.

from

${PROGRAM} ${REGEXP} '$(cat ${INP_FILE})' > ${OUT_FILE} 2>&1

to

${PROGRAM} ${REGEXP} $(eval cat ${INP_FILE}) > ${OUT_FILE} 2>&1

Answer 2

This is the error you get if you try to search in an empty directory:

$ tree
.
├── dir
│   └── file
└── empty_dir

2 directories, 1 file
$ grep foo dir/*  ## works, no error
$ grep foo empty_dir/*
grep: empty_dir/*: No such file or directory

Basically, when you run something like grep ./*, the glob (*) is interpreted by the shell which will expand it to the contents of the directory you gave. If the directory is empty, that expands to nothing and the shell returns an error. You will get the same error irrespective of which program you use:

$ ls empty_dir/*
ls: cannot access empty_dir/*: No such file or directory

So, I'm guessing that ~/5as-darbas/inputs/multiple_dir/ is empty. This is not a big deal and you can just ignore the error. If you want to deal with it more gracefully, you could give the directory name (no glob) and run a recursive grep:

$ grep -R foo empty_dir/

To do this with your current setup, change `~/5as-darbas/inputs/multiple_dir/* to ~/5as-darbas/inputs/multiple_dir/ and give grep -R as the first argument to your script:

$ myscript.sh 'grep -R' regExp/regExp tests/MULTIPLE.inp outputs/MULTIPLE.out

Answer 3

Ignore all of this, I was thinking the REGEXP was incorrectly safe guarded.

#!/bin/sh grep "$(cat $1)" $2

This simple case works with input files

tmp*

and

tmpfile

Called as

greptest re_file file_to_be_inspected