0

My input files are a single column with variable length of rows. My code is supposed to count those rows, and print a specified number. The problem I'm having, is if the number of rows captured in the variable "A" is less than 1, I want it to print at least 1 row so the output file isn't empty. If "A" (1% of the total number of rows) is greater than 1, I want to print that number of rows. My hybrid awk-bash code looks like this:

#!/bin/sh
for i in {1..2}
do
input="../ExpressionSet_"$i"_chunk.txt"
    for j in {1..2}
do

A=$(awk 'END{print NR*0.01}' $input)
Y=1
X=0
if (( "$A" -lt "$Y")); then
X=$A+1
else
X=$A

fi
awk 'NR<='$X' {print $0}' $input > "$i"_top1pc.txt

B=$(awk 'END{print NR*0.05}' $input) 
awk 'NR<='$B' {print $0}' $input > "$i"_top5pc.txt

Confusingly, I keep getting error messages like

thresholdSelector_pc.sh: line 20: ((: 0.24 -lt 1: syntax error: invalid arithmetic operator (error token is ".24 -lt 1")
thresholdSelector_pc.sh: line 20: ((: 47.24 -lt 1: syntax error: invalid arithmetic operator (error token is ".24 -lt 1")

By the way, inputFile1 has 24 lines and inputFile2 has 4724 lines. Thanks for the help!

  • 1
    AFAIK -lt isn't a valid arithmetic operator inside (( ... )) - and bash only performs integer arithmetic anyhow. Can you not do what you want within awk? – steeldriver Jul 14 '16 at 15:57
  • I tried to use "<" but the program reads it as a redirection character. – Greyson B Jul 14 '16 at 16:49
2

I think your root problem is that you tried to use bash syntax in an sh script. Bash defines extensions to the sh common denominator; if you want to use bash-specific features, your script must start with #!/bin/bash, not #!/bin/sh.

Sh doesn't have the ((…)) syntax for arithmetic expressions. But you don't need it here, you can use the portable [ … ] conditional syntax. In the [ … ] conditional, the “less than” operator is written -lt.

Braces {1..2} are another bash feature that doesn't exist in sh. Another bug in your script is X=$A+1, which sets X to a string like 42+1 if the value of A is 42; to perform an arithmetic calculation, you need to use the arithmetic expression syntax $((…)).

Also, as a general remark, always use double quotes around variable substitutions.

Another problem with your code is that it looks like A is a decimal number. Shell arithmetic only works with integers. I've adapted the algorithm but check what it does, I might not have rounded the way you intended. Using awk just to count the number of lines is overkill, wc -l is a clearer and faster way of doing that. Similarly, to print the first N lines of a file, just call head.

Yet another bug is that $i_chunk is the value of the variable i_chunk. To take the value of i and append _chunk, you need to delimit the variable name with braces: ${i}_chunk.

I have no idea what the loop on j is supposed to be doing, I've left it alone.

#!/bin/sh
for i in 1 2
do
  input="../ExpressionSet_${i}_chunk.txt"
  for j in 1 2
  do
    A=$(wc -l <"$input")
    Y=100
    X=0
    if [ "$A" -lt "$Y" ]; then
      X=$((A+100))
    else
      X=$A
    fi
    head -n "$((X/100))" "$input" > "$i"_top1pc.txt
    head -n "$((X/20))" "$input" > "$i"_top5pc.txt
  done
done

If you choose to write a bash script, you can make use of several bash features:

  • ((…)) for arithmetic evaluations (but still integer only)
  • typeset -i to declare an integer variable, so that assigning an arithmetic expression evaluates it
#!/bin/bash
for i in 1 2
do
  input="../ExpressionSet_${i}_chunk.txt"
  for j in 1 2
  do
    A=$(wc -l <"$input")
    Y=100
    typeset -i X
    if ((A < Y)); then
      X=A+100
    else
      X=$A
    fi
    head -n "$((X/100))" "$input" > "$i"_top1pc.txt
    head -n "$((X/20))" "$input" > "$i"_top5pc.txt
  done
done
  • You could go a bit further with the conditional operator: X=$((A < Y ? A + 100 : A)). – Lawrence Velázquez Jul 15 '16 at 2:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.