I have been writing/refining a relatively complex code for the past year, and learning bash along the way. My code is filled with functions that have complex multi-level while loops. I've recently been learning more about for loops and aside from c++ similarity and arguably easier to follow syntax, I am wondering if there are any pros/cons to using one vs. the other?

I am primarily thinking about execution time and compatibility with multi-level looping. Functions (5 total) in my script may iterate between 10,000-800,000 times each, the whole shebang taking 12-48hrs start to finish on a powerful machine, so I care about even very subtle time differences if reliable. I'm also interested in any other possible differences between for/while that wouldn't get mentioned in a typical forum/website due to their overall similarity, yet could affect a complex code in any way at all. For example, whether or not one finishes all processes (and releases memory) before looping and one doesn't? Or, interaction with c++ programs called within loops.

Here are simplified examples of two of my function code structures to demonstrate the multiple ways I'm using while loops:


var1="1"; var1max=${length}
while IFG=read -r var1lineFULL; do
    var1line=`echo ${var1lineFULL} | cut -f 1`
    var2="1"; var2max=${length2}
    while [ ${var2} -le ${var2max} ]; do
        sedvar=`sed -n ${var2}p < file.txt`
        wait; <c++ executable command>
        <more commands (bash/c++), if statements, echoing, verbose options...>
        let var2=var2+1
     let var1=var1+1
done < anotherfile.txt


var3="1"; var3max=`cat thirdfile.txt | wc -l`
while [ ${var3} -le ${var3max} ]; do
     while IFS=read -r var3line; do
          wait; <fickle c++ executable command>
          <bash commands.. if, echo, awk, cut, etc.>
          let var3=var3+3
     done < thirdfile.txt
     let var1=var1+1

Hopefully that makes sense enough to follow. Note: these loops work. I have them rigged to loop in exactly the way I need to increment integers that determine both how many times a loop runs and part the filename of the output created. They also create A LOT of lines of text, which I "3&>" into text logs.

So, what do you think? Is there any reason to switch to "for" when it wouldn't change the increment placing? Is there any reason not to? Would there be any time differences?

closed as unclear what you're asking by Thomas Dickey, Celada, don_crissti, Anthon, Jeff Schaller Apr 23 '16 at 21:20

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    It's not clear to me what you are proposing to replace with a for and for what reason. Also, since your variables are called var1, var2, and var3, it's not clear what the code is supposed to do. It seems like your code is also very dense and cryptic. Finally, have you considered codereview.stackexchange.com? It seems to me that is would be more on topic over there. – Celada Apr 23 '16 at 19:53
  • 2
    imo - if you are concerned with execution time - do not multiply call other programs (in loops). Rather implement what you need in one program (python seems suited (here) - it is easy and has a huge, convenient, standard library) and do not multiply call external programs. This 'process spamming' of yours is very resource intensive. Also, if you want to stay with bash, minimize calling 'outside' programs, for example echo ${var1lineFULL} | cut -f 1 calls cut (unnecessary) and could be rewritten with bash string substitution (tldp.org/LDP/abs/html/string-manipulation.html) – MCH Apr 23 '16 at 20:06
  • I'm voting to close this question as off-topic because it fits better at codereview.stackexchange.com – Jeff Schaller Apr 23 '16 at 21:20
  • Celada - I'm wondering if I should/could replace "while" loops with "for" loops. The code's purpose is theoretically complex and requires field-specific knowledge to explain so I tried to eliminate that aspect, sorry for any confusion. In this example, each "var" is an integer. – neurocoder Apr 23 '16 at 21:22
  • MCH - This is helpful, thank you! Using string substitution seems logical here. I do want to stick to bash, because (1) I know it, (2) when I open source this my target users (scientists, not coders) will probably only have basic bash proficiency. I can probably eliminate a few other outside processes as well. Not all, though. Given that I'm stuck with loops, do you suggest one over the other between "for" vs "while"? Also, would writing another script/function so that my loops are simpler and just call that one script/function make any difference, given it would also be written in bash? – neurocoder Apr 23 '16 at 21:33

If you know prior to the loop how many repetitions you are going to need, such as iterating a countable number of static files to process, you should use a for loop. That is what for loops are good at doing. If you are unsure, and the loop may be different based on what happens during processing, such as some sort of iterative aggregation algorithm that fills an accumulator to a certain value, or a search algorithm that will drop out of the loop when it finds the value that it is looking for, then use a while loop.

In your case, you should be using a for loop because you know how many iterations and are only incrementing your counter by a fixed value.

  • Nope, no complex algorithms of that sort - I want to run my loop/commands until every file has been processed. And increment is always +1. In my case I do not exactly know in advance how many files/lines because the initial input files, and what happens to them earlier in the program, determine how many files/lines I am working with. But, once I get to these loops, I'll have had the opportunity to calculate that number. Also, when I am reading from a text file, I can count the # of lines and get a priori information that way. But generally speaking, I will never know in advance. – neurocoder Apr 23 '16 at 21:13
  • I mean, in advance of entering the loop. Honestly, it is a pedantic style thing so that people reading your code in the future can get some intuition about what you are doing rather than a technical necessity, but you will make future coloborators, including future you, happier if you use the right stuff now. – Kevin Apr 24 '16 at 0:44

Not the answer you're looking for? Browse other questions tagged or ask your own question.