2

I am currently writing the below script. The code looks at a certain directory for a filename inputted by the user. The script first checks to see if the input file is a gzip, if so, it runs the corresponding checks. If the file is not gzipped it responds with an incompatible file text.

The issue I am running into is on line 7. No matter the file extension, I am receiving incompatible file as the final output.

#!/bin/bash
DATE=$(date +%Y-%m-%d)
L0_Report_Generator=("/home/ubuntu/$gzip_file")
echo -n "Enter File Directory:"$gzip_file 
read  $gzip_file
for gzip_file in {$L0_Report_Generator}; do
  if [[ $gzip_file = "test_sub"*"gz" ]] #Check file extension for gzip compression
      then
         gunzip $gzip_file
         echo "file Level 0 QC Check"
         echo ${DATE}
         echo "File Header"
         cat $gzip_file | head
         echo "Total Records"
         cat $gzip_file | wc -l
         echo "File Unique Records Size"
         cat $L0_Report_Generator | sort -u | wc -l
         rm $gzip_file 
    else [[ $gzip_file != "test_sub"*"gz" ]] #If file is anything other than .gz and csv - rort will not run
       then
         echo "incompatible file"
         fi
done
  • Since when has [[ required quoting? – Ignacio Vazquez-Abrams Nov 25 '17 at 5:14
  • the only times I've seen [[ require quoting is due to the use of wildcard which is why I used it in this script. – B.W Nov 25 '17 at 5:19
  • I'm not being interpreted where?? – Wildcard Nov 25 '17 at 8:23
1

If you want to check for a ".gz" filename extension using a wildcard expression inside of an if-statement then you would use an expression such as the following:

if [[ "${gzip_file}" = *.gz ]]; then echo true; else echo false; fi

Here is how you might test it out:

if [[ "file.gz" = *.gz ]]; then echo true; else echo false; fi

and:

if [[ "file.txt" = *.gz ]]; then echo true; else echo false; fi

The first example produces true as its output and the second example produces false.

Now let's look at your code. Your if-statement has the following conditional expression instead:

[[ $gzip_file = "test_sub"*"gz" ]]

In particular, you're including "test_sub" as a substring in your match pattern. Try removing that.

0

In addition to what @igal said about checking the file's extension, You have a lot of errors in variable syntax and usage. Start with line 3:

L0_Report_Generator=("/home/ubuntu/$gzip_file")

The variable gzip_file hasn't been set yet, so $gzip_file will be replaced by nothing when the shell expands it. Also, the parentheses in var=(something) assigns an array instead of a plain variable, and in this case that doesn't make any sense.

The fourth line, echo -n "Enter File Directory:"$gzip_file, has the same problem with the variable gzip_file. It also has the problem that echo -n is unpredictable and will do different things under different versions of the echo command. To print a string without a linefeed, you're much better off using printf "%s" "string to print", but in this case there's a better option that I'll get to in a minute.

The fifth line, read $gzip_file, appears to be intended to read user input into the variable gzip_file, but that's not what it does. In the shell, when you put $ in front of a variable name, that gets the current value of the variable. Here, you want to set it, so you must leave the $ off: read gzip_file. But that's not what I'd do. I'd include the prompt (which you echo on line 4) as part of the read command:

read -p "Enter File Directory:" gzip_file

Ok, now for line 6:

for gzip_file in {$L0_Report_Generator}; do

This appears to be setting gzip_file again (replacing the value we just read into it). Are you actually trying to set gzip_file here, and the previous variable references really should have been a different variable (maybe gzip_dir instead)?

Also, the in part doesn't make any sense. I think you're trying to use the variable L0_Report_Generator, but in that case the open brace should go after the dollar sign. But that doesn't entirely make sense either, because ${L0_Report_Generator} will (if I understand what this is supposed to do) just be the path to a directory. for ... in does not iterate over the content of directories, it iterates over a list of words, like for var in word1 word2 "word 3 which has several spaces in it" word4; do. If you want to get a list of files in a directory, you need to use a wildcard, like for var in dir/*; do -- the shell will expand the wildcard-containing file pattern into a list of matching files, each treated as a word, and iterate over them. You also have the option to limit matches to files with a specific extension by including that in the pattern, like dir/*.gz.

Three other notes: I recommend against using uppercase variable names like DATE, to avoid conflicts with the various all-caps environment variables that have special meaning to the shell or some utilities. Also, always double-quote your variable references (i.e. use "$var" instead of just $var) to avoid unexpected parsing oddities. And the else clause doesn't have a test, so using else [[ some test ]] doesn't make sense (and having then after else is a syntax error).

So if I understood what the script is supposed to do, I'd recommend replacing the beginning of the script with:

#!/bin/bash
date=$(date +%Y-%m-%d)    # Note lowercase variable
read -p "Enter File Directory:" gzip_dir
L0_Report_Generator="/home/ubuntu/$gzip_dir"

for gzip_file in "${L0_Report_Generator}"/*.gz; do

...And then (if the .gz pattern above is what you want), you don't need the if to check whether $gzip_file has a .gz extension, because the wildcard pattern will only list .gz files.

One more note: shellcheck.net is very helpful for pointing out basic errors in shell scripts. It misses a lot of what I pointed out, but caught the stray then (which I initially missed).

  • This was extremely helpful - addressing the above ultimately allowed for my code to begin functioning properly. A million thanks! – B.W Nov 29 '17 at 16:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.