6

This question already has an answer here:

The question may be trivial but with this simple for loop

for i in {1..10000000}; do echo "$i" line >> file
done

my shell crashed after some elaboration time (the tab in the terminal emulator was automatically closed) and file was not created.

It is GNU bash, version 4.1.11(2)-release.

Is this due to a memory limit or what else? And if yes which should be this limit?

With {1..1000000} it worked correctly.

marked as duplicate by Stéphane Chazelas bash Sep 5 '14 at 11:56

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

7

The reason this happens, is because the brackets are expanded before the command is invoked. Thus you actually end up with a command like:

for i in 1 2 3 ... 10000000 ; do ... 

.. and thus it uses up a lot of memory or it crashes.

The solution when working with long loops is to use the bash's c-style syntax:

for ((i = 1; i <= 10000000; i++))
do 
    ...
done

Or if you want to remain compatible with other shells you can pipe to a while loop:

seq 1 10000000 | while IFS= read -r line
do
     ...
done
3

My bash process performing this command takes a lot of memory, yes. 1.9GB in my case (x86_64). You might have run into your limit and the kernel out-of-memory killer has taken it out. For me, it works though. It takes about 3 minutes

 $ wc -l file
 10000000 file
 $ tail -n1 file
 10000000 line
 $ ls -hl file
 -rw-r--r-- 1 seb users 123M Sep  5 13:19 file

To confirm the hypothesis of RAM limitation: If you have swap, you can try to watch it being filled up. Use e.g. htop for this.

If I increase further to looping over 100.000.000 elements, the process eats also ten times more RAM: 18GB.

@user000001 gives an explanation and an alternative. I tested with the C style syntax (which he provided):

for ((i = 1; i <= 10000000; i++))
do 
 ...
done

This process then doesn't take more RAM than a regular bash process.

  • I have chosen the other answer because of the solutions proposed, but thank you for the very useful details provided about ram! If it was possible to choose two answers, I would do it. – BowPark Sep 17 '14 at 12:43

Not the answer you're looking for? Browse other questions tagged or ask your own question.